Well, I don't know you terribly well, but from what I do know is you're quite clever and very keen. Us guys are idiots are get nervous around that because we like to be the swooned after ones, and if the girl is hard to impress it makes us nervous. Not to mention you have strong opinions and not afraid to voice them, which can make some guys nervous.
Not saying any of this is bad! I'd definitely take a clever girl who can hold a conversation and articulate her reasoning over some dumb pretty thing any day, but I would be lying if I said that kind of girl makes me a bit nervous too.
Well, if you've never done anything just go at a pace that is comfortable for you; don't let him talk you into something you're uncomfortable with. If he won't like you because you're 'too slow' then he's a wanker and not worth your time.