I see your point, but you're skating the difference between the philosophical and the practical. We should all constantly self-correct for tendencies to thoughtlessness, no matter where we find them in our lives. That said, we should always be careful about accusing others (or assuming others) to be thoughtless. Especially in the racial arena, where comments can be very painful.
Remember, if you're worrying about whether or not you're racist, you're probably not. The real trouble comes from those who never think about it at all.