[It's an immediate response, without hesitation but the way he shifts in his seat or the fact that he's keeping his eyes away from the swordsman tells a whole different story. It's subtle but not undetectable - especially not for Zoro - and it's easy to see that the mosshead hit right on the mark with the comeback. So maybe he's not lonely, he's perfectly capable of making friends but never seeing his nakama?