I don't think its because its Stefan. I think its the fact you didn't listen to Rebekah's pleas. That Kol thought you would let her sell her soul. That you didn't care one bit for their feelings.
I think they want you to. They need you to do that. To care about what they think and feel. They need you to reach out to them this time.