He releases them so that he can A) get new Pokémon from the new region, or B) for story purposes. In fact, Ash actually releases very few of his Pokémon. The majority of them are sent to Professor Oak's lab (Bulbasaur, Tauros, Heracross, Infernape, Sceptile, Snivy, etc.)
Some go their own ways and are "released" but sometimes come back to Ash (Squirtle, Goodra, Lapras, Butterfree, ect.)
Finally, occasionally Ash leaves Pokémon in training. (Charizard, Primeape, Gliscor, etc.)
TL;DR Ash doesn't release many Pokémon, instead he sends them to Oak or leaves them with other Trainers. He leaves his team behind each region so that he can start fresh with all new Pokémon (with the exception of Pikachu and Aipom)
Here's some information on Ash's Pokémon's whereabouts.