You'd be surprised how much a person can do with even one of the stones. Alternate dimensions, timelines, future, past, they can all be turned inside out with more of them. I just don't know why. Anyone with that kind of power wouldn't need to remove us from the board to make their move, they'd just make it.