Some part of George still reacted very badly to seeing Padraig look so sad, and so he looked away to keep himself from snapping at Columba. It was good that someone could be relatively reasonable about Padraig, because that someone certainly wasn't going to be George.
He fell into step behind Padraig as the group headed inside.