Maaaaaybe because, like, you know what you're getting with Voldie? Yeah, he was pretty much evil but you see where it came from since he was conceived under a love potion and was an orphan and just had a bad childhood.
But with Umbridge.. she's like a sociopath. She's a huge fake. She'll smile at you and make your tea and slowly cut your fingers off and just keep smiling. SCARY!