OK, this is one of the weirder things for me, but apparently, when/if you go back home, you go back to the same moment you left, not remembering anything, but time goes on here. I think it has something to do with the same Infinity Stones that brought us here. Trust me, boggles the hell out of my mind.