Joe remained calm and quiet during the the silence, not phased by it. Again. Patience.
"The other gangs are moving in on our territory. There isn't enough loyalty among the lower ranks to keep them entirely in our pocket. I blame myself for that; I didn't spend enough time building it before you left."
He wasn't groveling for forgiveness. Just stating the facts.