I've heard some people say that the Tesseract has a mind of its own, that it will bring people on a whim, and send them home just as suddenly, with no rhyme or reason. In many ways, knowing that people that we are growing close to may be taken away and sent to their homes without possibly even remembering us, causes us to cherish everyone all the more. It is much the same with life and death, but here people can come back, never knowing that they knew us before.
I do, but I would give so much for a day spent with Diana, or an evening with Gilbert. It helps that one of my children is here, although he is here as an adult, so his memories are of an older and probably wiser mother.