You're not doing bad by him. You went to him, when you really didn't want to, remember?
I don't know that the consequences come from this timeline, though. This one is already insurmountably different from the one you came from. We just have to hope your mother shows up in this one too.
And of course you are affected by it. We all need our mothers in one form or fashion.