Her distress and her pain burned at him. It was respect for her, for his Padmé, that he hadn't forced the issue. Vader knew the path he was walking. He had turned this way once he had realized he couldn't be the man she wanted. Anakin Skywalker was dead and gone. Try as he might, Vader couldn't revive that spirit the way his son had, in the throne room of the unfinished Death Star.
And he had fallen, back into the savage and emotional path of the Dark Side. Back to where he could rage and hate, himself most of all. His failure to her. To his children.
But he could never lose Padmé. The connection to her was nearly a tangible thing, laced around his heart and through his soul. Her pain over the last several days had been like an open sore, rubbed with sandpaper. His own pain he could tolerate, even use to fuel his own passions, but hers left him weak. Numb.
finally, when he couldn't stand it any longer, couldn't cut himself away another time, he found her new hotel and knocked thrice at the door before opening it and stepping inside.
"Padmé," he said, his voice still carrying the ominous tones of Vader. "That's enough of this. You're coming back with me."