Any EV's in SP atk won't be put into ATK.
So basically, He would still have boosted SP Atk, even though He was using Physical Moves.
As you stated, The only thing that Switches around are the base Stats, not any boosts or EV's.
- It doesn't switch his Sp Atk and Atk, it Changes them
- Ev Training it in Special Attack would do it no Good when its Stats change to being higher in attack, unless you just wanted to keep using Special Moves, even though your Attack was boosted