"I didn't know that. I never really thought about it."
Also just stating facts, as if he's just discovering them, because he is. And listening, really listening, to what Spike's saying. He's the third person in two days to tell him this and it does seem to be about time he tried to address it.
"I'm sorry," he says softly. "I really hadn't thought about how I might come over like that. It's a persona I built up, so long ago - to protect myself, I guess, when I was a Time Agent and a con-man, to manipulate people and stop them asking too many awkward questions and fuck." There's his answer, isn't it? This is something he did deliberately, and it's so long ago and such a part of him now that he never even thought about how it might also keep away those he actually wants to be close to.
Rather blankly, he gropes behind him for the nearest bar stool and sinks down onto it. "Fuck."