Yeah but knowing that Robin was Jason (which I assume he did) and was therefore Bruce's ward. And then seeing Bats and Robin being father and son-like. It's a pretty natural assumption that Jason being Bruce's ward means that Bruce is Batman. Because who else could have been a 'father' to Jason. And I am pretty confident that Jim knowing Bats as well as he did (especially in terms of injuries) could always make the further link between "hey, everytime Batman gets seriously injured a couple of days later Bruce Wayne gets in a car accident"
I don't think that made sense, but I know what I mean.