He looks torn. There is truth in what Tony is saying. Going it alone will get him killed. Most of the reason he is allowing Sam on this journey is because he knows there's no shaking him. The other man knows what could happen and will stop at anything to save him. Not to mention the fact that unlike Steve he is qualified to help Bucky through whatever comes next.
Tony while determined has no reason to risk himself. That's why Steve insists desperately, "That's why I'm telling you to stay out of it. I deserve whatever happens next with him. It's my fault those things happened. I have to take responsibility for that without you trying to protect me."