He only came back because he hadn't thoroughly vetted Teddy. The blonde teenager seemed to know what he was doing with the baby true, but that didn't meant that Pietro trusted him with unsupervised care of Wanda's baby girl.
Although he was content to let him hold her and rock her.