If people get a certain idea about him... [it'll cause problems for him, possibly with the queen, possibly in general.]
[HE KNOWS that this place is more accepting in general than back home, but old habits die hard and it's put him extremely on edge that anyone should have certain ideas or expectations about the two of them]