You and he didn't have the best history. He and I didn't either, but there was a lot more to it. He did a lot of bad things, but he also did a lot of good ones. Losing him wasn't something I could do. I don't agree with Elena's solution, but I know how hard it was. She also hurt him more than he will ever say.
Just trust their opinions, and don't be so hard on him. Or me, maybe?