At least one of his brothers cared about the family and it's future.
"Very good. Also, as you have noticed, I have hired new servants. They have been told that Monsieur Dupont is not allowed in this house and you will take one of the family's servants with you when you leave. For now." Until he thought that he could trust his brother again.