The Doctor knew he wasn't fooling anybody with his over excited front - anybody who traveled with him for even a short amount of time could detect he was hiding something, and in this case it was his guilt over abandoning his friends and flying away without them. No matter how much he justified his action, the truth of his inconsideration still remained.
Once River played along, the Doctor knew what she was doing, but like actors in a performance, they remained in-character, all the while he knew he was going to get it, sooner of later.
Which was why the slap didn't come as much of a shock. The force of the blow caused the Doctor to swivel, then square his jaw to make sure it hadn't actually been broken. There was no reason to elaborate, he knew perfectly well why she'd done it, and didn't bother to defend himself because he agreed with her. He placed his hand upon the place where she came in contact with his cheek, and watched her pour.
"Are you going to throw the drink in my face?" he wondered. "It would be a waste of good brandy. I'd rather be slapped again."