Updated Study on Super-Shoes (Now Includes 2022 and 2023 Data)

Crazy that this comment gets no upvotes and a down vote.

It's kind of obvious isn't it? The fear of being instantly labelled a racist pig - even on an anonymous message board, far outweighs the acceptance of actually is happening in reality. Kind of mirroring American society on multiple issues not exclusively race.

Keith, I was reading through the code and noticed a minor error, I believe. The trend line is calculated using all the years, 2001 through 2023, as opposed to just 2001 through 2016. I hope you don't mind, but I scraped the data from IAAF just for the marathon and reconstructed the analysis.

You can see the difference isn't at all significant for the men, but a major difference for the women. In general, I don't think many of the men's distances will be impacted as most have a fairly shallow slope, but the women's races will likely be impacted as they've seen significant improvement in recent years relative to the trend.

P.S. The code fix was in defining your variables trendline_years and trendline_values.

docs.google.com

Elite Marathon Performance

Link:docs.google.com/document/d/1zVYJvG7yv-...

It appears you are correct. So glad you found this! I'll regenerate the graphs this evening.

It has nothing to do with racism. It is a comment on the complexity of the issue. There isn’t one thing like shoes. There are a half dozen variables you need to think about. Simple things like maybe a dozen people running WC instead of a major can change curves.

the OP doesn’t think a 2 min drop in about 5 years is a big deal. I go man that is fricken huge. Now if shoes are getting much harder. Will new foams and running get another 2 mins? Who knows.

Great take, Salvitor. Spot on. W45, it sounds like your squeaky mouth could use some WD40. This is great work and should be commended. The reality is that with potentially unlimited variables and micro-variables affecting myriad athletes across a multi-decade span, there is no testing methodology that could control for each. The author’s study points out this weakness explicitly and does not make any 1-1 causal claims. On the contrary, the value of this study is precisely in loosely correlating super shoes with improvements in elite race times. We are able to step back and look “big picture” at what macro-trends have emerged and consider whether a “2%” hypothesis neatly fits the data at face value. The fact that it does not only strengthens the conclusion that running is affected by many variables beyond shoes. The ultra-simplistic “super shoes have ruined running” claim thus needs to be re-evaluated and likely rejected. Shoes help, but they did not single-handedly sound the death knell of fair sport.

All updated now. The new graphs highlight the difference between the men and the women even more, and the text has been updated to reflect that as well. As you pointed out earlier, generating the trend line using a method different than a simple linear regression, like a time series, might well yield a different slope and result. Readers should keep that in mind and focus on the trajectory of the primary (blue) line of Average Times in considering the graphs -- i.e., do the points that come after 2016 seem out of place, or roughly in line with previous years?

I'm surprised though. Maybe metabolic efficiency is generally more valuable to elite women runners?

I don't really ever post here but I happen to have something to contribute here.

It shows the performances (y-axis in seconds, sorry no time for making this nice) over the last 23 years for men and women for 10k, hm, m. The slider at the bottom lets you place the start point of the super shoe era. Then the linear fits in each chart get updated with the corresponding data.

Maybe this will give a perspective I haven’t seen discussed yet. As many have already said, or implied, a time series plot of average race performances through the years doesn’t say anything about the underlying mechanism that is driving that improvement, only that improvement is happening.

Again, credit to you for putting all the data together. However, what isn’t identified in the graphs, because it would be nearly impossible to do, is where the human performance threshold lies for each event. It could be that prior to say 2020, the men were pressing right up against that limit, while the women were still a bit off. So, while the trend line would seem to suggest the men, post 2020, are “in line” with what would be predicted, the reality could be that they would not have been able to maintain that progression without the super shoes.

A non-linear function can look linear when viewed over a short enough interval of time. That’s not at all to discredit what you've presented, just to say that a linear trend will not hold indefinitely. So, if the point of inflection isn’t (or can’t be) established, it will look like progress as usual, even though it is actually a shift in the paradigm.

P.S. I appreciate the credit on your write up, but that wasn't necessary. Those types of things happen to everyone, at some point. Great job again for putting this all together.

A basic regression is fine, we aren't looking for some complicated hidden relationship here.

Yes, clearly men were much closer to the limits before super shoes. You can literally see the curve flattening for men before. Men had been competing in these distances for much longer with much deeper elite fields.

And women are also clearly responding to the shoes more. The reasons for this are more speculative, but some of it is likely linked to the same point about limits. They respond more because there is room more to respond.

An additional hypothesis for women is that they have higher relative vertical oscillation due to slightly different bio mechanics. This difference is documented. And it is also well documented that the super shoe high responders are generally people with higher oscillation. The reason for this is also theoretical, but common sense says the impact of bouncy shoes benefits bouncy runners more in that stride length is greater magnified. There are studies that confirm this last point.

Great work by the data scientists on the analysis. Thanks for doing it and thanks for sharing.

I ultimately believe that the outrage over these products being the "death knell of fair sport" came down to the reality that in terms of performance running, the market and its consumers (us) had simply been starved of basically anything over the last 25 years that helped performance.

Other sports have tended to have a constant flow of product innovation, real or perceived, whereas all innovation in running was focused towards what makes up 95% of the industry $$$ which is casual, everyday running shoes meant for walking the dog or intermittent low intensity sports. Take golf for example - over the last 25 years there was just a constant flow of "stuff" - hybrid shafts, different weighting systems, adjustable lie and loft sleeves, full carbon fiber heads (and they weren't even new at all!). So because it's been constant, no single thing is earth shattering. In running it was crickets for such a long time and all of a sudden it was like boom, a shoe that Nike is telling the world a human will run under 2 hours in the marathon with (and he nearly did)? It was a situation of relativity and context of which there wasn't any - it was all emotion and sentiment.

Running is also one sport runners love to compare/evaluate across generations because of that perceived lack of variation in variables across generations. So we like to think a sub 30min 10km run today is the result of the same blood sweat and tears we put into running that sub 30min 10km 25 years ago and it matters so much because it's the only measurable we have. Going back to golf - the guys on tour might hit it 40 yards further now than the guys on tour in the 90's, but that doesn't always mean the ultimate measurable (your score) is going to be better.

Your final statement there is the best one and sums it all up - the shoes help. They do. But we have far bigger issues pertaining to the fairness of the sport than the 35mm of nitrogen-infused foam we lace to the bottom of our feet.

Allie Kieffer recently posted results of her participating in a study comparing her running in 3 of the top super shoes. The Alpha Fly 3 showed a 5.8% increase over the control shoe. The other shoes tested were the Adios Pro 2 (.8%) and the Vaporfly 3 (3.7%). That's is a huge benefit! Couldn't tell what the control shoe was other than it was a current model Saucony.

It's talking about super road running shoes over the distance of 26.2 miles. Super spikes have been proven to be less effective, but just for fun let's say they're equally effective. That's an 0.07% improvement over 1 mile. That's a .17 second improvement on a 4 minute mile. Not really much of a difference even at the elite level.

A glance at your data shows that men's marathon times had stagnated for several years before super shoes and started falling again during the super shoe years. The average time during the period of stagnation (2011-2017) is 2:07:30 which is only 15 seconds faster than the predicted NoShoeEffect time for 2023.

AvgTime NoShoeEffect
2001 2:09:57
2002 2:09:33
2003 2:08:50
2004 2:09:43
2005 2:09:49
2006 2:09:31
2007 2:09:02
2008 2:08:37
2009 2:08:12
2010 2:08:07
2011 2:07:45
2012 2:06:58
2013 2:07:33
2014 2:07:19
2015 2:07:42
2016 2:07:42
2017 2:07:28 2:07:48
2018 2:06:54 2:08:03
2019 2:06:08 2:07:47
2020 2:06:58 2:09:12
2021 2:06:02 2:08:21
2022 2:05:53 2:08:12
2023 2:05:27 2:07:45

Okay. Uhh and what, because Allie Kieffer ran better in her AlphaFly 3's that's the proof behind the numbers? I think what amazes me most is how people just absorb the numbers without any contemplation of circumstances possibly around them.

First of all what's the 5.8% "increase" in? vV02max (velocity @V02max?), velocity at sub V02max? An arbitrary running economy metric? Secondly what are the details of the test - did she do it all on the same day? Maybe she did the test in the AF's first when she was freshest? Maybe she simply liked the look of the AF's more than the others which gave her a completely unquantifiable placebo effect benefit before she even ran. Maybe they did the testing over a number of days - was her biorhythm the exact same day of the test? What time of day was it?

My point is, there is a massive difference between a lab test on a robot and lab test on a living breathing human whose state is constantly changing and this will always be the issue in testing these shoes. Not only do we have responder theory but also we have to take into account factors even at the human level irrespective of their level of response.

And those results would validate what I said above - 5.8% "increase" in AF's vs 0.8% in Adidas Pro when fundamentally there is no difference in the makeup of the shoes apart from a small difference in foam type (not 5% different) and a useless forefoot air bag? Those numbers are meaningless.

Most of the improvement in the men’s and woman’s times between 1995 and 2010 was the huge influx of Kenyan and Ethiopian runners. You went from a handful of them to agents flying in dozens to every major race and for top guys to start skipping out on the track. Look at the improvement in the US where participation and doping are somewhat constant

2001 3 guy sub 2:15

2007 5 guys

2010 5 guys

2011 4 guys

2014 9 guys

2015 3 guys

2016 5 guys

2017 2 guys

2018 3 guys

2019 23 guys

2020 22 guys
2022 15 guys

super shoe effect is pretty dramatic right? Took 2 years for everyone to get them and figure out how to train in them and then the results were nuts. Now there are details like I think the iaaf excludes aided races and things like OT mess things up. And the sample sizes we are looking at are at the level where a race like Chicago have good/bad weather can make a a half dozen runner swing…

in the early 90s people plotted up charts like the OP and said woman would be faster than men by the mid 2020. After all the 5k for example had only dropped like 25 in 30 years and that was with track improvements. Woman on the other hand had taken about 90s over that time period. You need to think about all the variables involved in a problem or you draw trash conclusions

NCAA male track data. Super Spikes introduced 2021, fully adopted by 2022 outdoor season.

100th NCAA Outdoor Track Performance
Year 1500 5000 10000
2023 03:42.47 13:49.80 29:05.27
2022 03:42.11 13:46.80 29:04.04
2021 03:43.15 13:52.11 29:18.77
2020 season cancelled
2019 03:45.62 14:03.28 29:32.21
2018 03:45.22 14:03.58 29:42.61
2017 03:45.44 14:04.27 29:43.43
2016 03:44.93 14:02.40 29:42.32
2015 03:44.61 14:04.40 29:46.66
2014 03:45.33 14:05.11 29:38.94
2013 03:45.68 14:06.41 29:47.66
2012 03:45.79 14:04.37 29:47.91
2011 03:46.34 14:07.59 29:56.68
2010 03:45.98 14:04.39 29:51.28