To Zacharias, Merlin was like some skittish creature, and it was as though he had his wand pulled on him or something, something to frighten him. Even if he was just sitting there, being relatively harmless. Whatever it was, something about him scared the younger man.
And something about Merlin intrigued him.
"You can go," he told Merlin, and he didn't get up (even though seeing him to the door... well, balcony door where he'd left his broomstick would have been the polite thing - but he didn't want to accidently startle him more).
"I'll see you around," he said, a bit wryly, giving a little wave of a hand as he watched Merlin beat his retreat.