Nor should it have. The Surfer wore "silver briefs" at the time due to the Comics Code, but by the late 80s they'd been eliminated because of the simple fact that the Silver Surfer is as anatomically male as a Ken doll. He has no nipples, no genitals, no anus, no reproductive or excretory systems at all, and has a purely energy-based metabolism. He's not human and is physically asexual.
Which is one primary reason he and Nova were such a perfect pairing, as she was as physically asexual as he was and yet both of them retained their once-biological instincts they could no longer act upon. They could form the basis for the pairbonding they needed psychologically without facing the incompatibility they would have to deal with in dating a partner for whom sex was still physically possible.
I miss that pairing. Damn Ron Marz for killing her off. :(