He almost lost you once. She almost lost him once too that meant. So, he reached over to take her hands in his, letting her know that he was right here. That for all their trespasses, they worked together and that wasn't something he would blindly abandon. And he needed to hear it from her, as if that would somehow make it more bearable.
"What did I do?" While the other concern was what did she do, he still trusted her to have made what she thought was the right decision at the time. And really what she did was less important compared to what it was that she made disappear - what did she take away from him? "What did I do that would have taken me away?"
For as crazy as he might seem, he actually knew that some of things that he did or thought about weren't considered appropriate or even acceptable. Although he understood that as part of a limited world view that most people had, but he still knew that it was important to not share everything he did. He walked a fine line.