It was a sensitive topic the Doctor didn't like speaking about himself. He was a terrible father and couldn't imagine Jack being any worse than him. He tried to make it up with his granddaughter Susan, but the damage had been done.
The Doctor couldn't help to chuckle with him, "I was a bit of fun back then, yeah?" He reminisces momentarily about some of his adventures. "But I was hoping for something better." Beggers can't be choosers.