It's probably meant as a compliment to him or a reassurance but to Ianto it just sounds like an attempt to make him feel guilty over the way things started with Spike all those years ago and that just makes him more upset. "So glad to hear you tried out everyone else before deciding you liked me best. You really know how to make a guy feel special, don't you?"