"Everyone has a birthday," the Doctor returned. "No matter what you call it, but it doesn't really matter if you don't have anyone to share it with. He rarely spent years with the same person, each birthday would just bring up memories of the ones before and who he had spent it with.
"I prefer not to spend my time dwelling on my past." The past in general? Sure, fine, but his own life was better lived not looking backwards.