"Mmm. Well, that's not entirely true," Zacharis said, almost ruefully. "Because you wouldn't give me a chance to get to know you." And it was that which had frustrated him - not Merlin himself, not the person that Merlin was. The glimpses he got of that, rare and infrequent as they were, those he liked.
"I don't like that you made up your mind about me right from the get go. Or that you won't give me a chance - a proper chance - but that doesn't mean I don't like you, Merlin." He probably wouldn't have gone to all the trouble of the dates and everything if he hadn't!