Gems have a larger power increase. They also have the added benefit of activating an ability such as Unburden. They are also useful when used in combination with the move Acrobatics.
By the way, there is something interesting here. The Japanese name for Unburden translates directly into "acrobatics".