One difference is the version exclusives.
Black 2: Weedle line, Magby line, Bonsly line, Heracross, Plusle, Volbeat, Spoink line, Registeel, Latios, Buneary line, Stunky line, Gible line, Gothita line, Vullaby line, Zekrom, Black Kyurem
White 2: Caterpie line, Mime Jr. line, Elekid line, Pinsir, Skitty line, Minun, Illumise, Numel line, Regice, Latias, Glameow line, Solosis line, Rufflet line, Reshiram, White Kyurem
In post-game, in Black 2, you have the opportunity to visit Black City and the Black Tower. However, in White 2, they are replaced by the White Forest and the White Treehollow.
Reversal Mountain looks different. In Black 2, it's cool and watery, but in White 2, it's hot and filled with lava (comparison image here).
Opelucid City changes appearances. In Black 2, it appears more futuristic, while in White 2, it's more rustic (comparison image here)
Route 4 also looks different between the versions. In Black 2, new buildings line the sides (shown here). In White 2, they buildings are cracked and old (shown here)
In Black 2, you receive the Challenge Mode Key, Steel Head Key, and Black City Key. In White 2, you receive the Assist Mode Key, Iceberg Key, and White Forest Key.
Benga gives you a shiny Gible in Black 2. In White 2, he gives you shiny Dratini instead.
Funfest Missions differ between versions. Some Join Avenue shops are different depending on the version you play. The game aesthetic changes a bit between versions (for example, when entering a new area in Black 2, the notice telling you where you are is black. In White 2, it's white.).
Sources: Experience, this, and this.
Hope I helped!
