I actually am not sure if Brian was abused physically. Except for the occassional push and shove, I don't think Jack hit him, or Joan, I always had a feeling the abuse was psychological. In canon Joan tells Brian she protected him, let Jack beat her instead of Brian, and he does not contradict her, so there's some canon to back that.

The surprise at hitting Michael might have been more because it was Michael, of all people.