"It's not a bother to me to be there for you. You are a civilian." He acknowledges that he's going above and beyond with Iron Man. Tony's not a soldier as he pointed out before. There are different rules for him. He is allowed to bleed unlike Steve. "But, it's different when you're a soldier, Tony. I can't afford to have these moments when other people are around."