Well possibly he could have gone from protector to having a crush as Harry started looking like an adult... no reason to figure he has to be sexless in thought - after getting to really know Harry and Harry aging it would be *natural* for feelings to change.
It could be worse, he could sneak into his bedroom AND scarily sparkle in the sunlight...