It took the man's comment for Sam to realize that most of the churches he visited were empty. If not, they became empty very quickly--for one reason or another. Reaper, haunting, demonic possession... the possibilities were quite literally endless. Hence the difficulty of the job.
Sam would be disconcerted to realize how easily he could be read by a stranger. Some things he could hide from those who knew him best, even Dean, but those were the things that were written clearly on his face when he thought no one was watching. For example: it was obvious that Sam's usual intelligence was put to better use than religious theory and self-recrimination. Right now he was thinking quickly and logically, turning over ideas in his mind and then dismissing them as irrelevant or filing them away for further reference, a process clearly defined in the way his eyes moved when he wasn't seeing anything. Yes, no, possibly, or perhaps...?
"You're from another planet?" It wasn't quite disbelieving, but it was close. Sam said it the way older children stared at you when you talked about cows jumping over the moon, as if any moment you're going to say, just kidding. He recovered to answer the man's question with what sounded like a carefully constructed response, something he had been over plenty of times, perhaps in a place just like this. "This is the City, a metropolis taken from what appears to be hundreds of different dimensions and peopled by various inhabitants of those dimensions, some from different timelines and locations than others. It's been theorized to me that the City is sentient, and so far, as crazy as it sounds, I think that's likely. It appears to choose who it would to inhabit the City, and for how long, thought not by any criteria I've been able to discern."