Harry's little grin widened as Neville answered, which confirmed, indeed, that he was talking to Neville. He would have expected as much, but it was better to be safe than sorry. In this case, then, he mostly had to worry for Neville's safety rather than his own.
"You, too," he said, relaxing somewhat. "How've you been?"