[For a second he doesn't know how to reply to Dean. He's right. Sam did enjoy the kiss in the dream, and that was part of the avoiding. But he knows it was probably more about getting along with Cas while there was friction with Dean or not wanting to feel alone in his choices.
But then Cas answers and Sam is surprised by the sudden, forceful departure, before looking at Dean.]