I suppose this is a great new addition added to X and Y.
I also experience this as well, so it doesn't seem to be a glitch occurring to only you.
Game Freak have done a lot of things to experience mechanics this gen, so I assume this is just one of them.
Hope I helped. :)