It is simple, really. He is a dark figure on the edge of the Joker's vision, just barely taunting him with each step, and it takes so long to make it to the end of a single chapter of their deadly dance that he barely has time to savor the way the rubble falls at his feet before he is contemplating how he will make the Bats and his friends crumble.
Take away his lady friend and his man in the office, grumbles the bad guy in his head. He is still a little bit unsteady from the guzzled alcohol that the last memory works into his veins, and so the clown has worked his way into Jonah's limbs and fingertips and eyelashes and Jonah is dizzy and sore and scared.
The Joker is never scared. He can practically feel the pathetic little blonde man cowering inside his own skull and it is the perfect environment for a broken soul to rot away. He is proud of himself and he can practically feel the bile rising up in a man's chest as he searches for the woman that he failed. The woman he loved and let down.
Even when he recalls sitting in a jail sail, he knows that he was exactly where he wanted to be. He barely has time to fall in on himself before another memory teases Jonah's brain and it is time to retreat.