I suspect that the "size doesn't matter" people are being deliberately dense and there's no point debating them. But maybe they're trying to make a point about statistical anomalies.
For example, the high school my sons have run for is one of the smallest in their league. But they have been league champions in cross country for most of the past six years and have won about half of the track championships, i.e, they've performed way out of proportion to their size.
But they aren't a seven man cross country team. Typically they get 40-60 guys to come out so their talent pool is equal to or bigger than that of larger schools. Occasionally they have more athletes than the football team has. Bigger schools in the league often have smaller teams than my sons' school has.
So, yes, other factors can come into play which will create statistical anomalies. They've got good coaching, great team chemistry, and now a fairly long standing tradition of success which many kids at the school want to be part of. But you can't use them to "prove" that "size doesn't matter." If you eliminate our school and look at results for the other schools in the league you see that generally the bigger schools are more successful. At this year's indoor championships my kids' school won comfortably. The 2-3-4 schools were the three with the biggest enrollments.