Oh, I'm rather certain it was taken by force. If you were charged with protecting an Infinity stone and handed it over willingly, you're not just cryptic but kind of a horrible person and honestly, I don't get that vibe.
I literally sacrificed myself and trillions of other people so yeah, kind of doomed.