"Are you telling me to behave? That's like asking you not to do any hero-ing."
Not that Zemo particularly saw his own behavior as villain-ing. But he did recognize that some of his actions brought harm to others, so an argument could certainly be made for it. Zemo also picked up on Tony trying to shoo him away. Which didn't come as a surprise. Zemo hadn't really come down there to talk about the weather, anyway. But there was one more point he wanted to make before he left. He corrected his posture, his eyes resting on Tony.
"Just keep in mind that if you ever do have the itch to press my face into the floor, Sergeant Barnes will not be too pleased about it." Which was.. something of a threat. But it was important to Zemo to inform the others from his world that he was not completely alone on the island. He did still have an iota of power. With that, Zemo withdrew, letting Tony return to his work.