Cornell researchers: Vaporflys improve men's times by 2.1 to 4.1 minutes in marathon, women improve between 1.2 and 4 mins

Armstronglivs wrote:

https://sportsmedicine-open.springeropen.com/articles/10.1186/s40798-020-00250-1

"The use of the Nike Vaporfly/Alphafly running shoes does not generate a historically unusual level of performance improvement when evaluating the world record improvements and when using a performance improvement index."

lobster wrote:

Who in the top 10 in 2018 wasn't wearing the vaporfly?

I would certainly expect a HUGE percentage of the world top 10 in 2018 to be wearing nike as the shoes are giving them a huge advantage. But some quick googling reveals that 2 of the top 5 in the world were adidas.

1 2:01:39 WR Eliud Kipchoge - Nike

2 2:04:00 Mosinet Geremew - Nike

3 2:04:02 Leul Gebrselassie - adidas

4 2:04:06 Tamirat Tola - adidas

4 2:04:06 Asefa Mengistu - nike

4 2:04:06 Lawrence Cherono - nike

7 2:04:08 Sisay Lemma - nike

8 2:04:15 Berhanu Legesse - nike

9 2:04:37 Mule Wasihun - nike

10 2:04:40 Solomon Deksisa

Armstronglivs wrote:

889 wrote:

To think that new shoes do not help in the marathon is like saying new clubs and balls don't help in golf. But if you never played golf, you would never know.

So how does a new golf club help you run faster? Oh, right - it's a silly analogy. I might suggest new shoes in golf or new tees might be a more appropriate comparison - and would make very little difference. But I would be guessing - like you.

Not sure I understand this or where a connection between golf clubs and running faster came from (even in the context of trying to disprove of something conceptually) because there are two truths at play here regarding both sports and one underlying fact to both.

1) Golf club and ball technology has made seismic shifts in the improvement of players of all levels (just as a simple example go and google how the pro V1 with its solid core and urethane cover changed golf forever).

2) Running shoe technology, most notably in the last 3 years has created a similar seismic shift in improvement that has also happened amongst runners of all levels

3) Both have proven quantifiably. It's fact mate, end of argument.

Running shoes are to runners as a golf club is to a golfer - it's the primary implement/piece of equipment needed for the athlete/player to complete the physical requirements of the sport. Running shoes are not the equivalent of golf spikes/shoes and tee's dude. Come on man.

Armstronglivs wrote:

The analogy does not work. It depends entirely on the nature of the technical advance. Some will make a big difference, some very little. That new clubs have made a big difference in golf does not establish that new shoes will do the same in running. As others have pointed out here, it would require research that was able to isolate the shoes from all other factors contributing to running performance. I dont think the study that is the subject of the thread has done that. I referred to another study earlier in the thread which suggests the effect of the shoes is being exaggerated, and they have not made as big a change as many are claiming.

There doesn't need to be any connection between golf clubs and shoes. There doesn't need to be any connection or establishment of anything.

Research has been done that isolates shoes from other contributing factors with respect to running performance - I work in a company that does/has done it?

I understand exactly what your sentiment is and I do agree with it in principal because I very much have the same opinion that a large component of athletic performance is attributed to psychological and qualitative measures that we can't put numbers to. Of this there is zero doubt. However, I can tell you for an absolute fact there are quantifiable attributes to these products that improve performance and there we know exactly what they are and to what degree they help (also with a spectrum/range dependent on ability and physical characteristics of athletes).

Sorry dude.

Cant say much for Elites but Trust me Improvement is around 6- 8 Minutes if your time is around 2:50: 3:00

This is a pretty interesting analysis; though I'm not yet sure I'm convinced you've hit the mark, you've certainly put something out there to think about.

Two quick comments:

1) You said: "We would like to include an age effect in the model, to account for the fact that runners generally improve during their younger years and get slower during their older years." I'd re-think your assumption unless you can confirm it's true in your sample. I couldn't tell from the data in your paper, but couldn't you get some idea whether this is true by looking at the results for a runner who was in the same type of shoe in both races? Anecdotally, four famous recent examples (Sara Hall, Keira D'Amato, Marty Hehir, and Noah Droddy) would argue that marathoners don't necessarily get slower as they get older.

2) In your paper, you said: "We also considered including time-varying runner effects, to allow each runner’s fitness to improve or decline over time independently of the fitness of other runners, but the time- varying runner effects did not significantly improve the model or change the estimates of the Vaporfly effects, so we omit them here to streamline the definition of our model." It's somewhat difficult to believe that runners (serious ones) aren't improving over a four-year training cycle that runs from circa one Olympic Trials to another, especially when they may be on the cusp of an OTQ time. Are you saying that accounting for potentially increased fitness didn't change your model? That seems a little hard to believe, as well. Or are you saying it was overshadowed by some other effect?

You are missing a quite simple point. Technological improvements in one sport do not equate to the same kind of improvements in another sport in an unrelated form of technology. Each form of technology can only be judged on its own merits and the specific effect on that sport. There is nothing that says shoe technology in running will have the same quantifiable effect on performance as a technological change in a golf club (or tennis racket). There is no similarity between the sports or in the technology used.

Again, advances in technology in one sport will not mean that advances in a different form of technology will have the same quantifiable and proportionate effect in improving performance in another sport. Different sports, different athletes, different technologies.

The debate about shoes in running is just how much effect do they have on performance. There are barefoot runners who have succeeded in international competition - Abebe Bikila and Zola Budd come to mind. If they could win without shoes it suggests shoe technology may not be as big a factor as some think.

There are also a lot of variables to be taken into account to reliably quantify just what difference a difference in shoe technology makes to performance. Did the athletes train harder, were they fitter and in better form, was there a placebo effect etc. All of that needs to be taken into account. There are studies that say the latest technologies have not changed the sport more than any previous developments. If the facts on the new shoes were settled there would be no debate of the kind we see here.

How do factors like weather & course factor in with the 2-4 minute improvement for men? I see you're testing runners on Boston & Chicago but those courses aren't the same. The weather wouldn't have been the same. Training wouldn't have been the same.

Is the 2-4 minutes just from, say, a 2:40 without the shoes and then a 2:36 with the shoes? What is the control for being in better shape or getting a better weather day or running on a faster course or pacing etc?

2-4 minutes feels like quite a natural progression as someone learns how to run the marathon.

There were a couple questions about changes in fitness and effects of age:

On age: my conjecture is that when we select runners based on whether they exceeded a performance criterion, we are biasing the sample towards people who are in their prime, since a random runner is more likely to exceed the criterion when they are in their prime. Then when we follow the runners' performances forward, we are biased towards people who are past their prime. Obviously this will not be true for every runner in the sample. I'm talking about an average effect, and at this point, it's a conjecture that could be studied further.

On fitness changing over time: The published model includes a latent "ability" for each runner. This ability does not change over time, which does not account for the fact that runners' fitness changes over time. We tried using a model in which their abilities were allowed to change over time. This model has some nice features. For one, I captures the time-varying fitness. Also, suppose we had two runners that switched from non-vaporfly shoes to vaporfly shoes, but the first runner's marathons were 3 years apart, whereas the second runner's marathons were 6 months apart. It seems logical that the second runner's performances are more valuable than the first, because their fitness was less likely to change by a large amount in 6 months, rather than the first runner's fitness over 3 years. The time-varying fitness model allows us to put more weight on the second runner's performances when estimating the vaporfly effect.

However, the problem was that there was at least one runner who ran marathons in different shoes within a few weeks. The model was putting too much weight on this runner's performances, so we opted to go with the simpler model with unchanging abilities.

NERunner53 wrote:

How do factors like weather & course factor in with the 2-4 minute improvement for men? I see you're testing runners on Boston & Chicago but those courses aren't the same. The weather wouldn't have been the same. Training wouldn't have been the same.

Is the 2-4 minutes just from, say, a 2:40 without the shoes and then a 2:36 with the shoes? What is the control for being in better shape or getting a better weather day or running on a faster course or pacing etc?

2-4 minutes feels like quite a natural progression as someone learns how to run the marathon.

The model includes a term for each individual race (and each individual runner), to account for the fact that some races are faster, some are slower. The estimate of the vaporfly effect triangulates all the information we have about race difficulties and runner abilities, accounting for them as best as we can, to separate the effect of the shoes from the effect of race difficulty and runner ability. So it's not just a simple average of marathon times in vaporflys versus average of marathon times in non-vaporfly shoes. We try to account for other factors.

But who ever said that? I read the posts here I don't think anyone was directly relating one sport definitively to another?

You can't discount a statement such as "these shoes have had the same effect on the running industry as graphite shafts and steel headed clubs had on the golf industry" just because the tech wasn't the same and the impact not exactly the same. Of course they aren't - it's impossible to measure that. But that is a statement of first principal/sentiment - but surely you know this.

NERunner53 wrote:

Is the 2-4 minutes just from, say, a 2:40 without the shoes and then a 2:36 with the shoes? What is the control for being in better shape or getting a better weather day or running on a faster course or pacing etc?

2-4 minutes feels like quite a natural progression as someone learns how to run the marathon.

They aren't running the marathon 2-4 mins faster. They are running 2-4 mins faster than what is expected. Imagine that that you have 2 runners who both ran the same marathons and were the same age and so on.

a) runs 2:20 and a year later runs a 2:17 in the same shoes

b) runs a 2:20 and a year later runs a 2:15 in the supershoes

For A, we assume training (or conditions. Maybe they ran boston on a day when the wind was blowing) made them 3 mins faster. Why did B improve another 2 mins over A? The hypothesis is those 2 mins should be credited to the shoes. With only 2 runners that is sort of a sketchy assumption as things like training matter (maybe b was coming back from an injury for the first race or started running 110mpw instead of 90) . But as you get a bigger sample size, that stuff sort of averages out.

On their own, I am always suspect of these type of these studies as it is easy to mess something up or generate some thing from data hacking. But this is one more data point to go with lab tests on efficiency and more ancedotal stuff like HR at a given pace that suggest we are looking at a 2-4 min improvement.

So who exactly funded the study?

I mentioned a placebo effect of the shoes aware that this could be considered a bit off the wall . . .

https://www.bbc.com/future/article/20200501-the-performance-enhancing-trick-to-being-a-better-athlete

I'll put it even more simply then: comparing changes in running technology with those in other sports doesn't tell us anything about what effect the shoes have on running performance (if any). It is irrelevant. Only a study of the shoes and performance will do that. And so far the results have yet to be definitive.

really?... wrote:

Jeez, what rock have you been living under? For the millionth time, the 4% number is for reduction in "energetic cost," i.e., reduced oxygen consumption to maintain a given pace (improved economy), not a 4% improvement in performance. That translates to 2% or thereabouts in race performance.

animaxcg wrote:

So assuming this is correct can't Nike be sued for false advertising of the 4%+ running efficiency benefit? or do they have some * for legal reasons on the shoes marketing? because 2.1-4min over 120+ min is less than 4%

Jeez, what rock have you been living under? all the things you mentioned are "running efficiency" lol

Very interesting research. In the Upshot study on vaporfly, they didn’t only analyze vaporfly, but created ranking of different shoes. Everyone took note that vaporfly was leading these rankings, but they all seemed to overlooked the rather close second, which was Nike’s streak. It would be more interesting if this study also analyzed other shoes. People attribute the gain from vaporfly to its carbon fiber plate and foam, but this wouldn’t explain the seemingly performance gain from other shoes without these technologies, which means. If we don’t have an explanation for this, it’s could mean the seemingly performance gain is effects of confounding factors, as it’s hard to eliminate these effects when we are not conducting an experiment.

joeguinness wrote:

Hi Rojo, I'm one of the authors of the study and can answer any questions you or anyone else has.

We put the code and data online here:

https://github.com/joeguinness/vaporfly

One of the nice things about the researchers.one publishing platform is that experts can read the paper for free and leave their feedback, comments, and suggestions. We'd love to hear from runners and coaches.

peekay wrote:

First of all, Rojo did you know that you were cited in the discussion section? Congratulations, you can put that on your CV when you go for academic promotion! Maybe you just started this thread to toot your own horn...

I had no idea. And the study is like several months old right? I was just doing the homepage one night and saw some referene to it and put it up. I was amazed their was a Cornell study that I didn't know about considering I spent 10 years of my life there but there isn't a lot of crossover between departments and what not at large schools like that.

Hell, one of the study's authors didn't even email us to put it up on the homepage and he's an LRC regular.