Context.
At the time the US had been at war with Japan for five years and the concept of PC didn't exist yet. Racism was a huge part of the conflict between the US/Japan. (On both sides, Japan was one of the most racist cultures on Earth at the time.)
I've read a lot about Patton, if he was fighting the Germans he would have said "Krauts" or Italians "Wops". Also at the time, it would barely count as a racial insult. It'd be like calling a Canadian a "Cancuk."