Taken from the Anime, paired with logic from proven ecology:

Magikarp, during breeding season, lay over 1000 eggs. If all of these eggs hatch (Even just half of them, that's 500-1000 eggs from one Magikarp. Now, if you think of the 250-500 Female Magikarp from the previous generation, multiplied by 500-1000, you'll have 125 000 - 250 000 Magikarp during ONE breeding season.

Year after year, you'd slowly get more and more Magikarp. Sheesh, no wonder there's so many...