Doesn't matter if they were officially together or not, if Aidan has understood any of the complexity that surrounds Mal and Inara's relationship then he knows that it must hurt like hell. Him and Helix weren't together either, not really but being left behind without a chance to say goodbye still felt like a betrayal.