Only gamefreak can give a perfect reason for this, or they just did that for no reason at all...But since this is NOT the answer you want, I'll give you my best guess.
It appears that Graveler is a living, bipedal boulder. Its skin becomes tougher when it evolves. It also has four arms, but mainly uses the longer two to lift things, using the shorter two as a defense against attackers.
Golem Red/Blue description
Its boulder-like body is extremely hard. It can easily withstand dynamite blasts without taking damage.
So my conclusion is that when Graveler evolves to Golem, its body gets so hard that the 2 arms for defense becomes unnecessary, causing it to lose it during the evolution