"I think I should leave before we both make another decision we might regret." Quinn clearly regretted what had happened with Puck, he knew that already, it wasn't a surprise, but if he and Quinn did this now it would just be another thing she regretted. She might even hate him for it later.
And Rachel? Well, he didn't really regret that, he was a little worried because she'd been extra clingy, but he didn't regret it. Maybe he would later, maybe she would. He knew he hadn't been making the best of decisions lately and this was probably the smartest thing he could do right now. "Bye Quinn," he said heading toward the door.