Yes, and I would wager he doesn't want to erase the ones he sort of cares about from existence so that might be what's stopping him. That, and he's apparently collecting Old God souls like Pokemon thingees. I'm not sure if anyone has ever killed a deity - the Old Gods seem to have essentially just massacred each other, but I don't know the history or religion well enough to say for sure.
I did, I soothed him with lots of petting of the naughty kind!