That's not a bug, it's how the system works.
When you comment on a question / answer it's going to be related to that post. So if the comment was still kept than it would be moved somewhere else where it would make little sense, thus being useless keeping it.