I tend to think that plays a big role in it, but it's been implied that he and Bats never got along. I seem to recall him mentioning more than once that he wasn't very close to J'onn or Arthur either. I tend to think he just isn't drawn to people that come across as aloof and distant. All his close friends (Barry, Ollie, etc) are very open and upfront personalities, they seldom keep secrets, and you often know what they are thinking right off the bat, kind of like Hal himself.
Aquaman is monarch, he and Hal had little to connect on, J'onn is a nice enough guy, but often did come across as distant (and there was the entire 'keeping secret files on the JLA thing'), and Batman...well, he's Batman. He basically epitomizes 'cold, calculated and secretive professional distance). I can imagine Hal finding that deeply annoying (kind of how a lot of people suspect the shy person in class thinks he or she is better than everyone and just too stuck up to say anything).