This has been suggested before, and I'm going to tell you the answer is no.
Users with 20 points can have grudges with people, blah blah blah, there is just too much bias. There are so many reasons why this could not work.