If you do this, the votes will not take affect. When you refresh the page they will be gone. But, still...
>Say a person on Pokebase just hit 100 points and they say "Oh cool now I can downvote! I will go spam downvote a good question!"
Well eventually a moderator or Pokemaster would remove the unfair down-vote. So that might be covered.
>Cause I once had 100 points then went and downvoted an answer so now I have 98 points but the green arrow to downvote it not gray
Well, this is because the system doesn't show it as grey. This is so you can remove your vote, probably.