The problem with forbidding saying "you're racist" is that it would effectively shut down the opposition to racism. If we want to support combating racism, we have to be more accommodating to the person who is identifying/combating racism than we are to the person who is being racist (even if unknowingly).
As moderators, it should be our responsibility to provide those who "may be ignorant of offensive thoughts/words/stereotypes" with information and/or resources as to why what they've said/done is problematic, and take the appropriate disciplinary measures if necessary.