Yes, there is a big difference. Bulbapedia says of legendary Pokemon:
One, 伝説のポケモン densetsu no Pokémon, literally "legendary Pokémon", refers to Pokémon such as Kyogre and Groudon, who are featured prominently in the legends of the Pokémon world.
Later, from the same source:
The third and newest term is 神話のポケモン shinwa no Pokémon, literally "mythical Pokémon", which refers to the Pokémon which are spoken of as the creators of the Sinnoh region.
The term "mythical Pokemon" has later been used to refer to a myriad of different species, all of which are event-exclusive or were once event-exclusive. It is debatable whether mythical Pokemon count as legendary Pokemon; Bulbapedia notes that in Japanese media, they are generally termed as being distinct.
