HP
Walking at Rusturf Tunnel: Whismur (1 EV)
Surfing at just about any freshwater pond: Marill (2 EVs)
Super rod at Dewford Town or Slateport City: Wailmer (1 EV)
Super rod at Meteor Falls any room that requires waterfall: Barboach (1 EV), Whiscash (2 EVs)
Super rod at Victory Road: Barboach (1 EV), Whiscash (2 EVs)
Walking at Desert Underpass: Ditto (1 EV if not transformed), Whismur (1 EV), Loudred (2 EVs)
Physical attack
Walking at Mt. Pyre 1F-3F: Shuppet (1 EV)
Super Rod at Petalburg City: Corphish (1 EV)
Super Rod at Route 118: Carvanha (1 EV), Sharpedo (2 EVs)
Physical defense
Walking at Magma Hideout: Geodude (1 EV), Graveler (2 EVs), Torkoal (2 EVs)
Special attack: There's unfortunately no good place for this in Emerald. The closest one I know is Route 113, with Slugma and Spinda, but there's still a small chance of encountering a Skarmory that doesn't give special attack EVs.
Special defense
Super rod at Abandoned Ship: Tentacool (1 EV), Tentacruel (2 EVs)
Speed
Any rod at Route 104: Magikarp (1 EV)
Walking at Seafloor Cavern: Zubat (1 EV), Golbat (2 EVs)
Surfing at Victory Road: Golbat (2 EVs)
Walking at Altering Cave: Zubat (1 EV)
Walking at Artisan Cave: Smeargle (1 EV)
source