"I don't want a relationship with anyone new!" he flares, because has Ianto really understood this so little? It was being in a relationship - a relationship that could potentially last for-fucking-ever - that was the problem in the first place! A relationship with a pair of vampires who still acted like newly weds even after twenty years and couldn't understand why he still needed space to play the field.
"And I don't want to move, either." Even if the apartment does feel echoingly empty without Ianto, he's still not ready to leave it. That would be admitting to the finality of it, to the fact that there really is no way back this time. "You were the one who moved out in the first place! Why didn't you move further away?"
He knows why: Spike was already ensconced next door, but why throw logic in when it just gets in the way?