There are thousands of planets inhabited by trillions of people, not just human colonies. The war engulfed the whole of galactic civilization. Imagine World War 2, but on a galactic scale with dozens of alien species fighting a single enemy. I know the Batarians were almost completely wiped out.
Earth lost a billion people the first week, and a million people a day for the duration of the war. Other planets were just as heavily attacked.
I spent most of the war relieving pressure on our allies so they could freely add to our fleet to retake Earth.