It isn't to exclude human beings so much as to provide a safe place for my people. The X-Men are happy to work with human beings and superpowered humans, but due to a long and sordid history of persecution, we like to have retreats and meetings from time to time where the guest list is more restricted. It helps my people feel more comfortable, and it's by far safer given that so many of our younger mutants are still learning how to control their powers.