Well personally, I'd say the Pokemon is turning their back to you (their trainer) as a symbolism of the fact they're leaving for a different one. So it's like they're 'looking' at their new trainer rather than their old one.
^Just a logical assumption :P
You can always ask Gamefreak :D
