I thought there was never like a 100% agreement on exactly how Brian was abused, as we're never given a definite answer in canon (at least, that I can recall) If so, though, I definitely agree he'd be surprised and then a little disturbed by it, as it seems he's not physically violent with anyone else in his life, so for him to actually hit someone has got to be a big deal, especially if he put up with it growing up and probably decided he wanted to be nothing like his father.
Also, to comment on this, I scanned through 301 and ended up having to watch it all, including the ending, with sad, broken Brian and the Justin-hooker. And now I am sad and a little broken from having to watch that end scene *sniffle*