Damon liked that Elena was weary of him. He wasn't going to hurt her. Not unless he had a reason to. She hadn't done anything wrong and it wasn't her fault she was a doppelganger either. Damon just liked that she was a little unsure of him.
"She's in her room." Well she was outside of it but Elena didn't need to know that. "I saw her in the hall and she got called back to her room." He turned, looking at her with his intense eyes. "Don't worry. I'm sure she'll be along soon."