Fury
It's different here. Even if you had your suspicions that something was up prior to this who could've guessed it was HYDRA? On top of that Pierce was around to smooth over anything that may have looked questionable. It's different when they aren't around. When you start to notice something's wrong and the person you thought you could trust isn't around to say something. You start pulling at threads they'd rather you avoid and it all unravels.