It's quite alright, truly. It's a common misconception, and one we bought into when we were students as well.
The general concept has to do with personality traits and motivations. Slytherins weren't sorted there because they are evil, but because they are cunning. That sort of thing.