there's a possiblity that he's upset they're not really THINKING, so much as yelling slogans at each other. They're not thinking about him as a symbol of what we all have in common, instead they see him as a powerful enough symbol that they can bludgeon the other side with him.