The Move Deleter's house is directly to the right of the Lilycove Department Store, shown below. Please excuse the image quality.
He's in Lilycove city, to the left of the Department Store with a sign out front of his house.
In a house to the right of the Department store in Lilycove city.