He said that you thought I had an attraction towards you and that it made you uncomfortable and put him in an unpleasant position.
Honesty is the best policy. And with that in mind, I will say this. Father or not, if he ever hurts you I will beat him. Soundly.
I am not mad at you and do not want you to break up with my father because of me. Should, and I do not wish this on either of you, you two break up, it should be for reasons between you not because of me.