I know you're not going to hurt me. The person that hurt her was... you're not that. You won't ever be that. Some memories that aren't even mine are not going to change how I feel about you.
And anyway, to be intimidated by you, I'd have to under the impression that I couldn't kick your ass. Which is just silly.