Whatever the Winchesters might think of an angel like Uriel, to Castiel he was still a brother, and his turning and loss was cause for sorrow, not humor nor bitterness. The angel looked back at Dean through Jimmy's honest blue eyes, and then he said, "Uriel had no humor." Like Dean might not know, and Castiel saw fit to inform him. Castiel wasn't threatened by Dean, nor was he angry or pressed for time. He let Dean haul him around, because Dean did that to make himself feel better about things, and Castiel certainly gained no damage from it.
Once her was back on his feet he pulled irritably to straighten out his coat. (Funny how he thought of it as 'his' coat.) "The buildings move," Castiel repeated. "Interesting. Where is it you wish to go?"