The large majority of individuals who can use magic in my world aren't human. In turn, those same individuals usually employ magic to bring harm to others. Humans with such abilities in turn try to fight the monsters and so on. In order to protect everyone else from being caught in the cross fire, there are a number of clans who use their abilities from the shadows.
So anyone who can use magic isn't thought very highly of, I'm afraid.