Sometimes, it's clear a person voted with a bias - 99% of the voters judged one way, and this one voter skewed the opposite, it's clear they are not voting fairly, and we consider that, too.
Goodness, I find that a little scary. I often find myself thinking completely opposite to fandom on occasions, and that could apply to my thoughts about a story! I'd hate to think a vote I might cast would be cancelled because of that. :-(