If your answer is deselected, you lose the +20 points you get from having the answer selected.
The author of the question also loses the +2 points they gained from having BA selected on their question, unless a different answer to the same question was selected in place of the old one.