One difference is the version exclusives.
Black 2: Magby, Magmar, Magmortar, Spoink, Grumpig, Registeel, Latios, Buneary, Lopunny, Stunky, Skuntank, Gible, Gabite, Garchomp, Gothita, Gothorita, Gothitelle, Vullaby, Mandibuzz, Zekrom, Black Kyurem
White 2: Elekid, Electabuzz, Electrivire, Skitty, Delcatty, nNumel, Camerupt, Regice, Latias, Glameow, Purugly, Solosis, Duosion, Reuniclus, Rufflet, Braviary, Reshiram, White Kyurem
In post-game, in Black 2, you have the opportunity to visit Black City and the Black Tower. However, in White 2, they are replaced by the White Forest and the White Treehollow.
Reversal Mountain looks different. In Black 2, it's cool and watery, but in White 2, it's hot and filled with lava (comparison image here).
Opelucid City changes appearances. In Black 2, it appears more futuristic, while in White 2, it's more rustic (comparison image here)
Route 4 also looks different between the versions. In Black 2, new buildings line the sides (shown here). In White 2, they buildings are cracked and old (shown here)
In Black 2, you receive the Challenge Mode Key, Steel Head Key, and Black City Key. In White 2, you receive the Assist Mode Key, Iceberg Key, and White Forest Key.
Benga gives you a shiny Gible in Black 2. In White 2, he gives you shiny Dratini instead.
Funfest Missions differ between versions. Some Join Avenue shops are different depending on the version you play. The game aesthetic changes a bit between versions (for example, when entering a new area in Black 2, the notice telling you where you are is black. In White 2, it's white.).
Sources: Experience, this, and this.
Hope I helped!