"I do remember that day, you know," he comments, mock-warningly. "I know at the time I didn't remember anything of this, but afterwards I remembered the whole day as if it literally was yesterday, just not having been myself."
Which, at least, actually makes a bit more sense now he realises his form here is just a projection of his mental self.