She just wasn't getting it - it wasn't that he wished she'd never come with him, it was just that he couldn't bear to lose her again. It was easier to push her away now that wait until he had no choice.
"Well, I can't do it. How am I meant to find any sot of closure if I just keep saying goodbye over and over again? What's the point in getting close to you again, just to lose you again? Might as well just say it now. Goodbye!" he told her, almost shouting it at her, but his voice cracking just enough to stop it turning into a yell.