So, I have been getting tired of speculation about how much faster the super-spikes make a runner; and decided to do my own scientific analysis of what's going on. Some say the new spikes don’t help much or at all (and time improvements are due to better training - particularly during the summer and fall of COVID), whereas others seem to think they are rockets on your feet. In light of the speculation, I set out to actually come up with an estimate based on sound, statistical (using two-tailed t-tests ), comparisons using NCAA 5,000 meter times. Spoiler alert, the shoes do have a real impact, but not as much as many of you might think.
First off, I need to explain a little bit about how I did this study. I have been thinking, for a while now, that NCAA distance runners are, on average over time, getting faster; however, this was merely intuition and I had no real data on which to base my intuition. So, I decided to take the first 30 times from the NCAA 5k lists for the years 2010, 2015, 2019, and 2021 (I did not use times from the NCAA regional or championships meets). I converted the times into seconds (13 minutes equals 780 seconds, 13:20 converts to 800 seconds, etc.) and calculated the average times for each year. I also calculated the standard deviations for each year. This is what I came up with (this is just the first step so strap in):
2021 – 809.93 seconds (stand deviation – 6.074)
2019 – 817.73 sec. (SD – 6.034)
2015 – 821.23 sec. (SD – 7.851)
2010 – 822.3 sec. (SD – 7.410)
Just going by the means, the average times (for our sample populations) have been getting faster; we will see in just a minute if this apparent trend is real or not (through the statistical t-tests). Of interest is the standard deviations, which are indicative of the data spread (in other words, the dispersion of the data with respect to the means). The standard deviations for 2021 and 2019 suggest to me that the “slower” runners in our samples are running faster (it isn’t just that the fastest runners are getting faster – improvement is being observed in all of the runners in our 2019 and 2021 samples). This leads me to my first question/hypothesis.
Are the differences in mean times between years significant?
In other words, are runners getting faster overall? The answer to this question is pretty fascinating. The following are the results of my analyses.
• The first step was to analyze 2021 times versus 2019 times. The difference of the two means (809.93 seconds and 817.73 seconds) is highly significant. Remember, we are using a p-value of p < .05, which is the threshold that is typically used in science to determine if two sample population means differ – in other words, is the difference significantly different (not just apparently different) statistically-speaking. From our two-tailed t-test, the calculated p-value was 0.0000058. In other words, it was way under our threshold for concluding that the two means are different. 5k times for 2021 were much faster than for 2019; however, this is not the full story and we cannot differentiate between the training effect and the shoes. We will get to that question momentarily but first we need to test the other years.
• My overall hypothesis, if you recall from the beginning, was that NCAA distance runners are actually faster now than in the past. I looked at this using the data from before the super-spikes. When I compared the mean from 2019 with that of 2010, I discovered that the mean from 2019 is very significantly faster (p-value of 0.011) than that of 2010. The mean for 2019, however, is not significantly faster than that for 2015 (p-value of 0.058) but the difference is very nearly significant. By the way (in case you need a refresher), a p-value of .05 means that we can be 95% confident that the two means differ significantly (for real - in the real world). A p-value of .05 is really rigorous. So, the mean of 2019 is, for our purposes, truly faster than that for 2015 and a lot faster than that for 2010.
• Interestingly, the means for 2015 and 2010 are not statistically significant as the calculated p-value is only 0.59. So, we cannot, in reality, conclude that 2015 5k times were faster than 2010 5k times.
My conclusion from these tests is that NCAA 5k runners are, in real terms, actually running faster now than they were in 2010. Now, with that important information, we can get back to how much time the super-spikes take, on average, off of current 5k times.
What is the impact of the super-spikes?
For this part of my analysis, I adjusted the 2021 mean by, in the beginning, 9 seconds (3 seconds a mile), 6 seconds (2 seconds a mile), and 3 seconds (1 second a mile). I then tested my adjusted 2021 means against the mean times for 2019, 2015, and 2010. These tests allowed me to hone in on the per mile estimated savings from the shoes compared with the potential training effect (which probably isn’t just from 2020 but has been developing over a number of years as we saw above).
• The 9 second adjustment on 2021 gives us a mean of 818.93, which is not only not significantly different from that of 2019 (the mean is actually slower) but isn’t significantly faster than the means for 2015 (p-value of 0.210) or 2010 (p-value of 0.06). I inferred from this that the shoes are not giving a 3 second per mile advantage to athletes.
• The 6 second adjustment gives us a mean of 815.93, which is not significantly faster than 2019 (p-value of 0.254). It is, however, significantly faster than the mean times for 2015 (0.005) and 2010 (p-value of 0.0006). I inferred from this that the advantage from the super-spikes cannot be over 2 seconds per mile.
• The 3 second adjustment gives us a mean of 812.93 and is very significantly faster than that of 2019 (p-value of 0.003). From this, I inferred that the shoes do provide a per mile advantage of at least 1 second.
• I then tested adjustment-values between 3 and 6 seconds and essentially came up with a value of 4.8 seconds (1.6 seconds). I came to this estimate by finding the point where the 2021 mean is faster than the 2019 mean at the p-value of 0.05, which is right at the edge of our really rigorous significance-threshold. So, the spikes are giving an advantage of no more than 4.8 seconds over a 5k (1.6 seconds).
• From all of this, I conclude, that the super-spike-advantage is somewhere around 1.5 seconds (or about 4.5 seconds for a 5k). In other words, Grant Fisher’s PB would be 13:07 (or so), Cooper Teare wouldn’t have an Olympic qualifying standard, and Morgan McDonald and Drew Hunter both would (assuming they did not run in super-spikes, which I don’t think they did).
That is my more-or-less scientific analysis of not only the super-spike effect on times, but also distance running-times in general. I think we can sit back and feel good knowing that NCAA distance runners (at least at 5k) are truly, veritably, running faster than they did a decade ago, or even only 5 to 6 years, ago. It isn't just the shoes. Runners are training better from high school, to college, to post-collegiate. So, ENJOY!