If I were to guess, I'd say he did it because he couldn't control the Hulk, and while he could've helped with Ultron, he wouldn't know what would happen, afterward, and the amount of damage that he'd do.
Oops. Then never mind. That's what Amadeus told me.