They might not be here in this building or even in this city, but they have to be somewhere. The government has a way of tracking who comes and goes, and if they're not announcing a mass exodus, and no one simply left without so much as taking their stuff, then they're somewhere. Whether that somewhere is some lair with a mustache twirling ne'er-do-well or another plane of existence or shrunk down and thrown into a front yard to fend off ants, I don't know. Call it physics or call it gut instinct; I just know they can't simply be gone.