It's possible that this is actually a terrible choice, and by the time he's halfway through the song he's starting to suspect that it's swinging that way because, however hard he tries to emphasise the playfulness, there's a whole load of longing seeping through.
In an attempt to hold onto the lightheartedness (true to Cole Porter, who was always lighthearted), when the musical interlude comes he holds out his hand to her, his whole poise making it clear he's asking her to dance.