Damon in ways had a lot to thank this fucked up place for. If it weren't for this place he may never have ended up with Elena. She may have remained happily with Stefan and never given him any chance at all. And it was worth the shit that this place threw at him like this in order to be with her. Of course, he'd never say that out loud, but he was sure that Elena already knew that.
"I've been called worse," he said, in an attempt to lighten the mood. "Mystic Falls," he said. "Not many people have heard of it."