Of course it does. You're a good friend. If it doesn't feel right to you then it's not right. Just kind of gotta go with the flow, you know? Did that just rhyme? Oh God.
If you like him then I would suggest trying to start it over with him and take things slow. Being clear about what you want doesn't hurt.