[It really wasn't meant to be more than an opening but it's easy to see how it could be interpreted as so much more and for just a moment Sanji feels a little bad over the choice of words. Maybe it's easier for him with Zoro around, he's not entirely alone and the Strawhats have always been almost eerily good at adapting to new surroundings... but Gin? It's not that simple for him.
There's a taste of defeat in his voice. A taste Sanji is starting to become all too familiar with.
Not that he'll call him out on it, of course. They're men and they deserve to keep whatever dignity they can so in the end he plays it off as nothing, attention kept firmly on the work in front of him and tone held light, casual.]
So stick around for a while. What harm can it do, hn? You gotta eat anyway.