You can't; it just keeps happening.
It pays off, because ultimately you get to trade Pokemon with the person and those Pokemon are not usually found elsewhere in your game. The traded Pokemon will also be holding rare items, like Rare Candy, Old Gateau, Lava Cookie & Ragecandybar. The Pokemon you're traded also depends on the gender of your player character.
Serebii - Dropped Item - here's a guide listing the Pokemon available & the places they will call you & you can call them.
