Yeah, it'd just look black or grey, usually. Possibly sort of orange, depending where the creature's sensitivity stops. It'd be exactly like you or I looking at a human wearing a black shirt - that shirt's giving off a little heat, so to anyone who can see infrared, it's got a color. We can't see infrared, so we don't see that. Mysteriously, the shirt remains perfectly visible to us.