I had planned exactly that, actually, and it will be possible fairly soon. (Although the user that voted you down had way over 100 points so it may not help in every circumstance.)
I also like Will's idea of encouraging people to comment why it was down-voted.