He was not above slugging Olive, particularly if she was smacking him with frying pans, dishes, furniture. She was pretty violent in her own right, and a few bad guys had to be saved from her.
Swee'pea was capable of knocking an adult senseless, too, but of course he was raised on spinach. I don't even want to mention Poopdeck Pappy, he thought punching people was like waving to them. The only cast member you could assume wasn't likely to hit you was Wimpy, but you were lucky to get away from him with a penny to your name.