No a swarm is not the only way you can get a Dunsparce in Generation IV.
You can get a Dunsparce normally or by using Rock Smash in Pokémon Heart Gold and Soul Silver in Dark Cave on the Violet City Side of the cave.
Note: There is only a 1% chance of encountering a Dunsparce normally in the cave.
Although Dunsparce has an 80% encounter rate when using Rock Smash to smash rocks inside the cave. If you do not encounter a Pokémon when using Rock Smash in the cave you may instead get an item. This would either be a Red Shard, a Blue Shard, a Green Shard, a Yellow Shard, a Heart Scale, a Revive, a Max Either or a Star Piece.
Hope I helped :)