"I was thinking of her when we were being kept there." Which was the truth. The long hours of lying there on the floor, helpless and left to simply wait to be liberated gave him a lot of time to think about her, if anything happened who would tell her, how would she have raised their son alone? "I didn't want to leave her. I don't want her to leave me now."
At the question of if he wanted her to stay until Audrey got there. That could be long, longer than he would want, and there was the issue of Oliver who, from what he had heard, was in pretty serious condition. But his need to have someone other than his mom won out. "For a while, but if it's too long, go back to your husband."