I don't think it's a huge deal as new comments only show up in the "All Activity" page, not on the home page or Questions page. Having said that, comments should be on topic and regarding the particular post.
So if the question was "Which Pokemon has the highest stats" and someone answered Arceus, a comment like "Arceus is my favorite Pokemon" that's pretty pointless. Or a comment like "This is a good answer" - the post should just be upvoted instead.
So pointless comments, yes please do hide those if you spot any, otherwise don't worry about it.