Elsa had every reason to be cautious about getting involved with him. While he was undoubtedly working on that awful track-record of his, Loki was still, for all intensive purposes, Loki. Most people couldn't handle that. She didn't seem bothered with who he was, all the more reason for his attraction, so it wasn't keeping him.
"Safe is dull and lifeless. Besides, life doesn't quite allow for safe. There's no such thing." Oh, he knew he was difficult and usually bordering the less than conventional. What did it matter his method if the end result was what everyone wanted to achieve, good or bad?
Playing along, Loki inclined his head forward, their distance becoming quite close. "I'd know better than to waste your time with anything less than genuine. In fact, I wouldn't waste my time either." He wouldn't ever put himself in such a vulnerable position, expel so much energy, to something he wasn't passionate about. One could say many things about Loki but when he put himself to something--he gave it his everything, a dangerous attribute if used negatively.