Supes probably recognized Alfred when he saw him again as an adult. Not to mention there were photos of the child Bruce Wayne in the newspaper.
And I'm sure Bruce must have seen some pictures of Clark as a kid as well (I remember there was a story where Bruce pretended to be someone else and talked to Martha Kent, though I don't know if that's still canon).