When it comes to returning these specific memories, I'm prepared to disregard that.
[Sebastian is always prepared to disregard morals. With that, he continues walking - Porthos may stop him or follow, but if he doesn't, Sebastian is done here. Ciel's memories are somewhere out there, and there's no point in standing around. He does turn his head in Porthos' direction, briefly:]
Had I known for sure yours was not the memory I seek I would have left it well alone. [....."Sorry about that"?]