At this point in his timeline, the last interaction Seishirou had with Subaru was at the end of the bet. During that confrontation Subaru completely crumbles...the way I read it though, it really seems like Seishirou is waiting for him to fight back. He drags it out, driving his point home trying to get a rise and goes so far as to say that if he really resisted, Subaru would be strong enough to break free of his spell. But Subaru...doesn't. I really think that Seishirou wanted Subaru to rise to the occasion and fight back, and was dissapointed when he didn't...which is why I play him, at this point, resentful towards Subaru's apparent weakness.