So, the X-Men have some sort of long-standing rivalry against the Avengers in your world, or general mistrust? I see the explanation for mistrust for any Avengers then, but I can also see how mistrust would be developed on our end towards them after that incident. Nonetheless, this isn't just an Avengers vs. X-Men thing when you take into matters that there are people from neither one of those groups, especially young children.
I'm honestly not really seeing a connection between "my husband has come back with no memories" to "let's build a safehouse in space", but if you say this is her being unreasonable, well... I believe it.