That's totally fair! We have just as much fun with the smaller group.
For the record, my D&D credentials include having played for about 14 years and DMed for 9 or so. It's been consistently DMing for the last two years, except for when someone else gets the itch and I actually get to play.