A statistical analysis of the effect of super spikes on NCAA 5K times (runners are actually faster, it isn't just the shoes)

The problem with statistical analyses is that you can provide a lot of detail and use sophisticated terminology that can give the illusion of authority/expertise even when the most fundamental requirements for the analysis haven't been met.

In this case your analysis procedure is founded on having two independent random samples. There is nothing random about taking the top 30 times. (This would represent a sample of what population?)

You also seem to be making the cardinal sin of assuming causality from an observational study.

Think about how you would run an experiment to estimate the effect of the spikes (matched pairs?) and how you would obtain your subjects so that you could generalize your results. Your analysis provides food for thought, but only to the point that it might motivate a proper study.

Your analysis is flawed.

This isn't a true multiple comparison situation.

There is either a difference or there isn't and the difference is meaningful or it isn't.

A better approach would be to randomly select D1 athletes and compare PB% this year to a previous year, or consider delta pb% either absolute or relative sense.

Or consider the slope of the trend-line or a moving average, looking for homoscedasticity.

Another approach would be plotting results using kinetics, looking km/Vmax and asymptotics

You could also come up w/ something using the Rand index.

Adjusted Rand index would be preferable

Does racing in shoes w/ carbon plates help predict whether an athletes is among those who ran a PB in 2021?

Probably redo this at the end of the season. The faster times from year to year are due to one thing: increased use of PEDs in American distance running and not some special training routine.

It does matter, but assuming everyone wears super spikes just makes the conclusion weaker, and you've already shown a statistically significant effect. If I'm thinking of it correctly, if fewer than all the runners have the special spikes, and you're still seeing this improvement in times, then it was would make the spike effect even stronger (i.e., the effect of the spikes on times is dulled by those not wearing them, if the spikes are making for faster times, assuming other confounding variables).