Sam didn't rise to the bait. After all, it wasn't like he'd done anything in the Impala that Dean himself hadn't done. And he had met Annie. The memory of that meeting came paired with the unpleasant memories of what he'd been like without a soul. The same went for the mention of showing Fred the hideout. The disappointment might have shown on Sam's face for a brief moment before he buttoned it up.
It wasn't fair to just expect that this could be another place he could hide and try to forget the things that he'd done when he was soulless. To think that he could simply lose himself in the memories of this place without bringing in people that reminded him so strongly of that time.
"A witch?" he repeated incredulously. He'd never known his brother to trust those who used magic.
Then again, they'd never been in a place long enough to put down as many roots as Dean had here. Sam was glad that he'd had that chance, but he did feel ever so slightly left out. There were people here who knew and cared about him but mostly because of the Sam that had come before or because they were connected through Dean. He didn't feel he deserved the regard for either reason really.
"Maybe we should explore a bit more before we decide who does and doesn't need to see the place. After all, if this belongs to some secret society, we might want to figure out what's in here before we throw our first housewarming party."