[He's trying not to show that the idea unsettles him just a bit. Not that he utterly dislikes Lacus. He doesn't. He never actually disliked her so much as felt slightly threatened by her relationship to Neil in the way a younger brother might be threatened by their older brother's girlfriend.
Now she... well, she wasn't CB, but she wasn't a threat either.]