I think his point is that even if you can have a good, lasting relationship, there's still the fear (and more importantly, the possibility) that the person you're in a good, lasting relationship with will be killed. He moved on and Gwen's still dead. If Ben got married and his wife were killed, Ben would move on and his wife would still be dead.
As good as the marriage with Mary Jane is (it's eventually coming back. It's just a question of when) and without getting into the question of who Peter loved "more," which is a silly question anyway, I have no doubt that if he were offered the chance to go back in time and save Gwen's life, but it would mean he never married Mary Jane, he would do it. And without thinking about it, I just came up what would have been a better way to get rid of the marriage than what they actually went with.