there's a fence in the way
Are you talking about the three houses sitting together at the bottom of the town?
If so, they are inaccessible. But they play no part in the storyline, so don't worry. :)
Hope I helped. :) Source