Humans get to know people through interaction and observation. First impressions are important for that reason. [He's been around the block more than a couple times.]
Misjudging someone isn't heartless. It's a normal thing many people do without realizing it.
Refusing to change your opinion you're proven wrong? That's a bit more heartless.~