My brother and father got involved with the Boy Scouts when my brother turned 11, so I started being involved in Webelos and BS around 8.
We did a lot of camping during the year as a family, and during summers we all went on a two-week vacation with the BS troop. The troop always made these family trips so there were a ton of people who weren't actually BSs. There were a lot of other kids' older sisters around so I was happy and most of the parents left us alone. There was a general agreement that all the parents were looking out for each others' kids and the kids knew they could get busted by any of the adults. If anyone did act up they were publicly admonished and humiliated in the process.
My brother managed to act a fool one morning and he was pilloried. He didn't want to get up one morning and one of the adults pulled my brother's tent down on him. My brother freaked out and went on a tirade. He was later made to apologize to the adult and all of us for it.
In addition to that we did various summer camps in other states, and also worked on the staff some times. I think I enjoyed teaching Archery the most. All the Nature merit badges were fun, but mostly just earned by rote--archery is a sport so it is more enjoyable.
Anyway, my experience was that family vacations are awesome, but they have to be done right. The nuclear family does not need to go on a vacation alone. They already spend a lot of time together. I think its best to do large, group vacations where the agenda is set by committee rather than your parents and you don't have to be around your stupid sibling more than usual.