That's because he's one of a very few vessel's that equal Messiah qualified. Some are pissed that it's a Winchester baby, because in your world that wasn't supposed to happen. Ever.
They felt the demon, that's all. Red flags went up. Alarms went off, and they couldn't to shit about it. They did not like that. Don't let those dicks get to you.