"Yeah, and my father was a cold heartless bastard who didn't do anything but buy up land and ignore his wife and son. That doesn't make me him. And you didn't do that, so I don't see why you need to play martyr to appease some crazy 'What If' you've got played out in your head!"
Unfortunately, Tony wasn't exactly the most stable of influences himself, and watching his best friend all but eviscerate himself with misplaced guilt was starting to wear thin.