With Bruce, there's the idea that she's an adult female crime-fighter in Gotham. It's not the best connection, but it still makes sense, particularly given Bruce's tendency to jump to the worst. Obviously I can't prove it, but I'm pretty sure that if Barbara hadn't got shot, Bruce would've been much less quick to make a connection.