Rufflet does only have a 5% chance of appearing on Route 23. If you get tired of running around through patches of grass, I suggest just trade for one, or wait until after E4 to complete your Pokedex.
There's that Braviary on Route 4 that appears every Monday, but then again you need to have beaten the E4 before you can go onto Route 3 and have access to the Day Care in order to breed for a Rufflet...