That's definitely a rule back in our reality. But here it's.. different? I've been told we're in a pocket dimension. Anything that happens here won't influence our reality or timeline, and if we ever leave, we won't remember being here.
Can't say I've experienced this myself, but Bucky has.