It's saying you don't want to risk fighting them, yeah. But having a clear-cut enemy in these games is a strong disadvantage. When everyone else has to decide between friends, if there's ever an easy choice, you don't want it to be you.
You knew that, right? So don't put yourself down for thinking that way. Is it impossible now?