[Luckily for Xander's privacy and for the entertainment of everyone else, Caliborn is not at all familiar with that reference.]
James human. We will meet again.
Fuck you. [It's a very conversational and casual "fuck you." Said in the same tone as most people say "goodbye." Extra context clues come from the fact that he hangs up right after saying it.]