He's not in the same CLASS as the Avengers, no, but he does have a superteam of his own working for him, and a LOT of power, influence, and information at his command. Just how long he can handle these things is another issue, but at the moment, he's certainly powerful enough and paranoid enough to take seriously. He may not have that unique relationship with the Avengers that he has with Spidey, but that doesn't mean he's an incompetent, and ANYONE who has enough power at their command should not be underestimated.