It’s because of the movepool.
It has access to many good special moves such as Draco meteor, shadow ball, fire blast, thunderbolt, And surf. It’s physical movepool just isn’t quite as good. It’s only physical STAB ghost moves are phantom force and astonish, which are quite bad, sadly. It’s physical movepool is OK, with access to dragon darts, psychic fangs, U-Turn, dragon dance, and sucker punch. It’s just not quite as good as the special movepool.