Almost all races don't consider other people of their color to be their race. I think only Eskimos avoided doing this, but that's mostly due to having bigger issues to worry about. Africa's been a bed of tribal warfare for longer then their recorded history. China, Japan, Korea, Vietnam, etc invaded each other constantly. Hell, Japan is one of the most oddly racist countries I know.
Regardless of color or race, people have always broken themselves down from language, to race, to creed, to country, to state, to county, to city, to neighborhood, to clan, then to family. And then to bits of family they like. It's a really dumb move by the Inhumans, because they've seen Earth fight and war over everything, even while being the same race.