Yes, there is. You need to find someone with the Rock Friend Safari in XY that has Shuckle in it. Shuckle you find there have a chance of having Contrary. Here's a tip: lead with an Intimidate Pokémon and see if its Attack raises or lowers. If it raises, it's Contrary.