Because the fear of mutants is based on them en masse, not the individual mutants. It's the fear of them 'coming over here taking our jobs and ruining your way of life', only instead of marrying your daughter, they'll be you daughter.
It's the fear of the future not being the domain of people like you.