'A while ago' works for me, same with 'in days past' or 'once upon a time, when Gotham was a much more simple place...'
But I'm more than willing to concede that you're right, it doesn't work if you want to look at it logically I just don't care, dammit!. Besides, I'm assuming that Damian is going to be revealed to be some sort of rapidly aged pod person created using scrap bits of Bruce's DNA and concentrated evil at some point, so yeah...