Maybe as a Magby, out of it's five egg moves, four of them are Physical moves. But because Magmortar's movepool itself has more Special than Physical moves, it switches to Special.
Also, Magmar came out in gen 1, which happens that all Fire type moves will be special. Magby came out in gen 2, with a contrast of a physical mon(or is it just me). Magmortar came out in gen 4 with a contrast of a special mon. Also, since Magmar at first, had higher special attack, Game Freak made it that it would be easy to trade over your old Magmar from gen 1 or 2 to gen 4 and evolve it without changing it's Physical > Special or Special > Physical sets