If you want to defeat Brawly, then you definitely don't need a Swellow.
0 Atk [level 15] Taillow Wing Attack vs. 0 HP / 0 Def [level 19] Makuhita: 30-36 (48.3 - 58%) -- 95.3% chance to 2HKO
source
If you're still sure you want a Swellow before defeating Brawly, then you can first have Taillow defeat everyone in the gym except Brawly. Then, go to places like Route 109, 110, 111, and 117 and use your Taillow to fight trainers there.
