We experience a lot here that should be impossible. I think it's more than possible for whatever traps us here to wipe the memories you had of this place if it chooses, and to make it feel like no time passed.
But I suggest talking to Hope Mikaelson. She disappeared from here for days, but claims that she was gone for years and still kept all her memories of what happened here and to her.