If you're dating anyone primarily out of a fetishistic attraction to their skin color - whether it's white, black or whatever - then it's questionable at best.
There's a huge difference between, say, dating a woman who happens to be Asian (or, to use your example, dating a man who happens to be black) versus dating them because you have a specific thing for their respective ethnicities, because in the latter instances, you're pretty much reducing them to the sum total of their skin color.