A lot of people outside the Circle aren't aware of phylacteries. Templars like to keep them quiet, considering they're a form of blood magic.
If I'd had them done after I became a Warden, I would've stayed. I think the Templars gave up on tracking me then. It would've been a little strange, though, after years of being around humans...