Harl says he never really loved her but then she's wistfully holding a photo frame that says "Harley baby love you forever --- J." What do you make of it?
It's just a bone he's thrown to her. Big ol' beef bones for Bud and Lou, declarations of devotion to Harley-- keeps 'em happy and (more or less) obedient. I think at the end of the day no one's going to really get in the way of his long-standing infatuation with himself (why do you think he hates being laughed at, or being the butt of the joke?).