Generally speaking, a right is something that every person has or should have (for example: freedom of speech is a right), but a privilege is something specific to one person or group of persons, which usually requires a person to meet a certain criteria (for example: passing a driver's test). It is one of those things like all squares are rectangles, but not all rectangles are squares; all privileges are rights, but not all rights are privileges.
In your definition of privilege, it says, "Right, advantage, or immunity belonging to a person, class, or office." That is, it's a right belonging to a certain person, class, or office. That is how you draw the line between the two.
Certainly though, anyone who thinks they simply have the right to drive will have to face facts if they have their license taken from them.