You remember correctly. (And without the pad and pen, he'd go nuts.) It's from the novel Last Son of Krypton by Eliot S! Maggin, long-time Superman writer. While it's sometimes guilty of the worst excesses of pre-Crisis Superman storytelling, it's still a neat read.