If you can't solder and can't help but shake it around, then I'm afraid you're out of options.
Seriously though, I think transferring the circuit board to a new blank cartridge might be best for you, granted if you can remove the circuit board fairly securely. The cheapest way would be to buy a used, crappy Gameboy game for really cheap, remove its circuit board, then put the Gold's circuit board and battery in the new casing.
Replacing the battery is a great way to preserve old Pokémon games, but it's very easy to crack or bend the "teeth" of the cartridge itself when removing it, and yeah, the shaking of the battery sounds like it's loose and not connecting right as a result.
That's all the advice we can give - we're a database, not a customer support forum. Try that out though, and I wish ya luck!