I don't see the big deal. It's not like he and Jo are biological (or even legal for that matter) family.
Plus it's not like he even originally recognized/recognizes Ultron as his son. And yes I know Pym responds when Ultron calls him "Dad", but your fighting him anyway, why not go along with it?