Where the Brojos failed at probability analysis of the Boston Marathon

Of course this is after the fact, but I think a better way to predict odds in this case is to look at the likelihood of different scenarios and then look at the chances that someone (Meb) wins in that scenario. Different types of ways the marathon could've been run:

1. Lead pack goes out WR pace and Meb runs with them - I'd assign him almost 0% chance of winning this type of race because of his PR. But I wouldn't expect this type of race ever only because he wouldn't be dumb enough to try to win Boston at WR pace.

2. Lead pack goes out WR pace and Meb holds back - By holding back Meb gives himself a shot if everyone else blows up and he's better than the guys who also are in the chase pack.

3. Lead pack goes out at reasonable (for 2:05-2:07 guys) pace and Meb goes with them. This would be the more likely expected race but it didn't happen. Pace maybe too hard for Meb here to win.

4. Lead pack goes out at reasonable pace (for 2:05-2:07 guys) and Meb runs with chase pack at a reasonable pace (say closer to 2:09). Meb has a little better chance in this case.

5. Lead pack goes out "conservative" (Meb's pace in last example) and Meb stays with lead pack. In this case I think the pack would win because they simply represent more bodies and the pace is "easy" for them all.

6. Lead pack goes out "conservative" and Meb takes off leaving them behind and wins. This is closest to what happened.

Now there's probably tons of scenarios that could happen but I'm trying to keep it simple. Anyhow, the difference in this type of analysis is it groups the lead runners together. Prior to the race would anyone have thought Meb would build a lead and hold off? Probably very little so even if we did this analysis beforehand we wouldn't have assigned high probability to scenario 6.

I actually like the wide odds range you pointed out. It shows how impossible odds are to calculate for a single occurring event in time and that it's so much easier to like myself analyze after the fact. Ventolin actually does this a lot and biases his findings to make himself seem like he predicted stuff accurately.

malmo wrote:

I'll repeat. Determining the probability of event B happening today is not the product of the complementary probabilities of two or more events that have happened in the past.

kartelite wrote:

I get what you're saying, which is a pretty fundamental point, but I don't think that anyone is really assuming those are actual probabilities of some event.

That's exactly what the Brojos are saying.

kartelite wrote:

It assumes homogeneity among runners, that the probability of "success" (sub-2:07) is the same for this race and in prior ones (the "sample"), i.e. you're making iid Bernoulli draws. Under these unrealistic assumptions, 71% may be the MLE of "success" for each runner at Boston, and then you could estimate the probability of 0 successes at 0.29^8 or whatever.

But that's not what the Brojos are saying. They've taken the success rates of each of nine runners and aggregated them, then applied a metric to them that would only be used for calculating the probability of the undesired outcome (complementary result) of a series of nine independent events. Boston marathon is a single event and the probable outcomes are incalculable.

Forget the arbitrary assignment of the 2:07 number and follow me here.

The logic of their “calculations” is that since the aggregate success ratio of sub2:07 marathons is 22/31 or 71% the failure ratio must be the complement of that number, or 29%. Now they go one step further, without ANY mathematical reasoning at all, and decide that the chance of NO ONE running is the product of this complementary number (0.29) multiplied by the number of athletes that make up this pool. That makes no sense at all, because Kimetto, Desis and Regassa are perfect as far as sub-2:07 marathons. So the three of them are part of that aggregate pool that gives a total of 29% 2:07 misses. There is no dependent relationship between their perfect 2:07 performance ratios (100%) and the other 6 runners that make up the group of nine.

What the Brojos are trying to do (without a math basis) is calculate the number of times that the group as a whole would not break 2:07 in 9 consecutive trials, as if the results of an athletic event is the result of the product of random independent probabilities. Once again, this is wrong..

Following their looped logic, after the first trial there is a 29% failure rate. No accounting for the perfect records of Kimetto, Desis and Regassa.

After the 2nd trial 0.29x0.29 = 0.0841 No accounting for the perfect records of Kimetto, Desis and Regassa. Al three would have run 2:07 or better

After the 3rd trial 0.29 x .029 x 0.29 = 0.02438 No accounting for the perfect records of Kimetto, Desis and Regassa

All the way down to the ninth trial 0.29^9 = 0.0051% No accounting for the perfect records of Kimetto, Desis and Regassa, and once again no mathematical rational for making calculations in that way.

SISO.

rojo wrote:

David Esptein, author of SPorts Gene, loved the math one.

There's nothing mathematical about it. You just grouped together the rates (R) of 2:07s by N runners, then took the complement (1-N), and multiplied that number to the Nth power, without any mathematical basis for doing so. Complete bunk. I couldn't believe that you felt it was worthy to post?

rojo wrote:

The two pieces you talk about are two of the one's I'm most proud of.

David Esptein, author of SPorts Gene, loved the math one. Of course it's not independent and we said as such. It's also not independent becasue like yesterday, everyone goes out slow or fast together.

We said that was way low.

The point of the piece was just to get you to think. Obviuosly when we have a range of 1 to 30 and to 1 to 19,000 - we aren't being scientific.

Also tilastapaja normally doesn't include DNFS which is what top Africans often do.

It's obvious you didn't intend for it to be "scientific". The fact you thought it had enough merit to even be a discussion starter is the scary part.

phd stats guy wrote:

...........

I actually like the wide odds range you pointed out. It shows how impossible odds are to calculate for a single occurring event in time .............

It doesn't show that calculating the odds is impossible, it shows that rojo didn't know it was impossible, dreamed up some funny math, and then put it out for the world to see.

Malmo,

I'm the first one to think that the subject of probability can even make smart people look uneducated. When they gave a range from 1/27 to 1/19000, that was my clue to ignore their model, as hype designed to set your expectations somewhat realistically. I certainly agreed with their point that Meb and Hall had little chance to win, and didn't require making my own estimates to agree.

It has to be said now that the "perfect" records of Kimetto and Desisa are now 75% -- not too far off now from the estimated 71%.

rekrunner wrote:

When they gave a range from 1/27 to 1/19000

I don't know much about probability, but I do know that anyone giving odds of between 27/1 to 19000/1 to a 2:08 guy is going to go broke some day.

You can make a reasonably accurate guess at the odds if you follow the sport without doing any math at all (at least not consciously). You might get something reasonably decent if you built a model and actually tested it first, but that's a lot of work.

In contrast, what would you have guessed Meb's odds were without doing any stats? Probably like 3-5%, and that's much closer to the "real" figure than 0.00-whatever.

I'm starting to think that the Brojos might consider shredding their doctorate degrees in theoretical mathematics (or was it applied math?) from MIT (or was it CalTech?) and starting from scratch - a basic stats course, even an online one from DeVry University, might be a good option.

You mean some of you actually read the nonsense in those articles they write?

I consider that highly improbable.

kartelite wrote:

I get what you're saying, which is a pretty fundamental point, but I don't think that anyone is really assuming those are actual probabilities of some event.

It assumes homogeneity among runners, that the probability of "success" (sub-2:07) is the same for this race and in prior ones (the "sample"), i.e. you're making iid Bernoulli draws. Under these unrealistic assumptions, 71% may be the MLE of "success" for each runner at Boston, and then you could estimate the probability of 0 successes at 0.29^8 or whatever.

I'm just happy to see a post from kartelite!

I just read stuff by the Brojos with the idea that it comes from the mind of a fan...because that's what it all is. Nothing wrong with that. They aren't journalists and shouldn't be held to that standard (proper analysis, proper writing style and grammar, etc.).