He wasn't a very pleasant man, was he? But I'm glad you took the potion. It makes me feel safer.
And I've already closed up the manor and Forter castle, so my answer would be yes. Rodolphus said his plans won't take place for another few weeks, but I'm not sure if I trust him on that point. I'm sure it will be fine, but remember the Cornfoots. It's up to you, but I'd leave now if I were you, tomorrow at the latest.
If you do go to the ball, do take care and leave if something feels odd.