Which is totally understandable, he's your family. There's nothing wrong with missing him. Do you know where he is? Is there a chance you could visit him? I'd go with you, if you wanted.
Ugh, HYDRA. They're the worst. But they're also master manipulators, and have survived for so long even though everyone wants to wipe them out (in most universes) anyway.