The big reason is the Pokemon tower which is where dead Pokemon go. Also there are 'ghosts' there before you get the silph scope. The ghost later become ususally haunters and gastlys which are ghost types and the ocassional cubone. The music there is also sort of creepy but the main reason is that Pokemon tower, basically the Pokemon cemetary is in that city. There is also a rumor that says that kids ages 7-12 commited suicide and had brain issues after hearing the lavender town music. This is fake but its still a creepy rumor.