I don't actually look at it like I'm suddenly missing from that world. I think of it more like... that world was a past life, or that world was another life. I feel like things bleed over, but I'm not that Dani. I'm this Dani. I think we kind of exist concurrently, if that makes any sense.
So maybe you aren't missing from there, maybe that's not something you have to worry about.
I sometimes wonder where the rest of my friends are, too, though. If they ever existed in this world. Some of them are gone forever, and some of them might never know who they were before.
I don't think it's wrong to think about that stuff.