I am afraid I have not examined all of the scientific literature on the subject. Alas, that is my downfall.
But I can only imagine that the fiery golems that dwell at the earth's core wouldn't take well to the gem eaters consuming the fruits of their millennia of labor.