It was actually been stated in story in a 1970's backup story.
Bruce Wayne is leading a camping trip with a bunch of underpriveleged kids, and they all tell campfire tales of what they think Batman is really like; supernatural being, high tech avenger, superbeing, etc. At the end, for fun, Batman himself jumps down from a tree to scare them. The kids don't miss a beat, but just point out to Mr Wayne that there's no way HE could ever be Batman. Bruce reflects that children, and those who have nothing to fear, AREN'T scared of a guy in a batsuit, because there's no reason he should.
Then there the silent black and white story about a Gotham kid needing a nightlight, and eventually finding that the batsignal works better.