they used to go out with joss's current girlfriend, which joss is probably cool with (99.99% of the time) especially because it actually looks like rory and joss have a whole bunch in common (outside of taste in women), and joss needs more friends to talk about cartoons and video games with.