Doping Prevalence in Track: a 2018 study found that up to 43.6% of athletes at 2011 Worlds were doping

orsilochus wrote:

Since Chepn'getich's suspension, I've fallen down a bit of a research rabbit hole on doping. In doing so I came across a very interesting study published in 2018 which surveyed athletes at two major international competitions - one of which was the 2011 IAAF World Championships.

The researchers randomly approached athletes and gave them an anonymous survey on a tablet, asking if they had knowingly violated anti-doping regulations by taking a prohibited substance within the last 12 months. They also tracked how long it took each respondent to take the survey.

The result: 43.6% of athletes responded that they had knowingly doped in the last 12 months.

Now, my initial reaction was one of skepticism in the methods of the survey. I'm still not sure I fully understand the study design (they used something called an Unrelated Question Model), but they go into some depth as to how they control for errors or unintended "yes" responses. Specifically, they looked at how the numbers change if they took out the top 10-50% of respondents in terms of how fast they took the survey - the thinking being that some athletes may have taken the survey so quickly they didn't actually stop to pay attention to what they were answering. "The prevalence estimates decreased upon deleting the 10%-20% of fastest respondents, but stabilized thereafter.

You can read the full text here:

Link:www.sportsintegrityinitiative.com/wp-c...

So assuming this survey is statistically sound and that no clean athlete would knowingly respond "yes"... why in the world would these athletes answer honestly? What's the incentive? The study was anonymous and randomized, but you don't dodge doping control by being honest. I'm still trying to wrap my head around that part.

But 43.6% is a lot. Especially when you consider that a significant portion of doping respondents were probably smart enough to not answer "yes." It's hard to not look at this study and think that the whole sport is dirty top to bottom.

rekrunner wrote:

orsilochus wrote:

Since Chepn'getich's suspension, I've fallen down a bit of a research rabbit hole on doping. In doing so I came across a very interesting study published in 2018 which surveyed athletes at two major international competitions - one of which was the 2011 IAAF World Championships.

The researchers randomly approached athletes and gave them an anonymous survey on a tablet, asking if they had knowingly violated anti-doping regulations by taking a prohibited substance within the last 12 months. They also tracked how long it took each respondent to take the survey.

The result: 43.6% of athletes responded that they had knowingly doped in the last 12 months.

Now, my initial reaction was one of skepticism in the methods of the survey. I'm still not sure I fully understand the study design (they used something called an Unrelated Question Model), but they go into some depth as to how they control for errors or unintended "yes" responses. Specifically, they looked at how the numbers change if they took out the top 10-50% of respondents in terms of how fast they took the survey - the thinking being that some athletes may have taken the survey so quickly they didn't actually stop to pay attention to what they were answering. "The prevalence estimates decreased upon deleting the 10%-20% of fastest respondents, but stabilized thereafter.

You can read the full text here:

Link:www.sportsintegrityinitiative.com/wp-c...

So assuming this survey is statistically sound and that no clean athlete would knowingly respond "yes"... why in the world would these athletes answer honestly? What's the incentive? The study was anonymous and randomized, but you don't dodge doping control by being honest. I'm still trying to wrap my head around that part.

But 43.6% is a lot. Especially when you consider that a significant portion of doping respondents were probably smart enough to not answer "yes." It's hard to not look at this study and think that the whole sport is dirty top to bottom.

This "study" has been discussed here a lot. The main flaw is that anonymity doesn't guarantee survey compliance, and the Unrelated Question Model doesn't estimate non-compliance (unlike other random response techniques). The study itself discussed about eight different ways the answers would be inaccurate, providing accurate mathematical models for each one, but with no way to estimate the extent of any of these inaccuracies. So behind the 43.6% is about eight unknowns.

Most world championship athletes are not there to take some survey on a tablet, and many just auto-clicked "yes" so fast, they couldn't possibly have read the survey instructions. Filtering out the fastest 30% reduces the estimate to 31.4%, but this still doesn't guarantee the remaining slowest 70% are accurate, just by virtue of being stable.

It also combines men, women, sprinting, distance, throws, jumps, and race-walking into one global figure.

The same researchers collected alternate survey data from the very same athletes at the very same event, and using a different model, Single Sample Count (published later in a different paper), they estimated 21% doping, with 30% non-compliance for the same 2011 World Championships. When you look at the extreme lowest to highest published values given for 95% confidence intervals, the values for doping estimates range from 9% to 47%.

Note also that 2011 also precedes the Russian/IAAF bribery/extortion scandal.

The conclusion of both studies is that the methods are promising, but further refinement is necessary to reduce survey non-compliance, and recommended that follow-up surveys with refined methods should be done. To my knowledge, this hasn't been repeated since 2011.

Does the UQM yield estimates that are too high or are the SSC estimates too low? In this commentary, we suggest that the analytic model used in the 2022 SSC paper (which we have termed the “P-model”) may underestimate the true prevalence of doping, and we show that the SSC data are consistent with a much higher prevalence of doping when using a plausible alternative model (which we have termed the “A-model”). In particular, the A-model yields a much more realistic estimate on the control question regarding the prevalence of supplement use than does the P-model. We also present a hypothetical scenario of 1000 athletes with a 55% prevalence of doping, where athletes report 75% of the actual number of “true” statements. In this scenario, the P-model would yield only an 11.2% estimated prevalence of doping, rather than the actual number of 55%—again suggesting that this model may not be the best for analyzing SSC data. At this point, therefore, we would suggest that methods for analyzing SSC data may deserve further refinement, and that the 2018 UQM results remain at this point the most plausible estimates of the frequency of past-year doping among elite athletes.

rekrunner wrote:

orsilochus wrote:

Since Chepn'getich's suspension, I've fallen down a bit of a research rabbit hole on doping. In doing so I came across a very interesting study published in 2018 which surveyed athletes at two major international competitions - one of which was the 2011 IAAF World Championships.

The researchers randomly approached athletes and gave them an anonymous survey on a tablet, asking if they had knowingly violated anti-doping regulations by taking a prohibited substance within the last 12 months. They also tracked how long it took each respondent to take the survey.

The result: 43.6% of athletes responded that they had knowingly doped in the last 12 months.

Now, my initial reaction was one of skepticism in the methods of the survey. I'm still not sure I fully understand the study design (they used something called an Unrelated Question Model), but they go into some depth as to how they control for errors or unintended "yes" responses. Specifically, they looked at how the numbers change if they took out the top 10-50% of respondents in terms of how fast they took the survey - the thinking being that some athletes may have taken the survey so quickly they didn't actually stop to pay attention to what they were answering. "The prevalence estimates decreased upon deleting the 10%-20% of fastest respondents, but stabilized thereafter.

You can read the full text here:

Link:www.sportsintegrityinitiative.com/wp-c...

So assuming this survey is statistically sound and that no clean athlete would knowingly respond "yes"... why in the world would these athletes answer honestly? What's the incentive? The study was anonymous and randomized, but you don't dodge doping control by being honest. I'm still trying to wrap my head around that part.

But 43.6% is a lot. Especially when you consider that a significant portion of doping respondents were probably smart enough to not answer "yes." It's hard to not look at this study and think that the whole sport is dirty top to bottom.

This "study" has been discussed here a lot. The main flaw is that anonymity doesn't guarantee survey compliance, and the Unrelated Question Model doesn't estimate non-compliance (unlike other random response techniques). The study itself discussed about eight different ways the answers would be inaccurate, providing accurate mathematical models for each one, but with no way to estimate the extent of any of these inaccuracies. So behind the 43.6% is about eight unknowns.

Most world championship athletes are not there to take some survey on a tablet, and many just auto-clicked "yes" so fast, they couldn't possibly have read the survey instructions. Filtering out the fastest 30% reduces the estimate to 31.4%, but this still doesn't guarantee the remaining slowest 70% are accurate, just by virtue of being stable.

It also combines men, women, sprinting, distance, throws, jumps, and race-walking into one global figure.

The same researchers collected alternate survey data from the very same athletes at the very same event, and using a different model, Single Sample Count (published later in a different paper), they estimated 21% doping, with 30% non-compliance for the same 2011 World Championships. When you look at the extreme lowest to highest published values given for 95% confidence intervals, the values for doping estimates range from 9% to 47%.

Note also that 2011 also precedes the Russian/IAAF bribery/extortion scandal.

The conclusion of both studies is that the methods are promising, but further refinement is necessary to reduce survey non-compliance, and recommended that follow-up surveys with refined methods should be done. To my knowledge, this hasn't been repeated since 2011.

orsilochus wrote:

Since Chepn'getich's suspension, I've fallen down a bit of a research rabbit hole on doping. In doing so I came across a very interesting study published in 2018 which surveyed athletes at two major international competitions - one of which was the 2011 IAAF World Championships.

The researchers randomly approached athletes and gave them an anonymous survey on a tablet, asking if they had knowingly violated anti-doping regulations by taking a prohibited substance within the last 12 months. They also tracked how long it took each respondent to take the survey.

The result: 43.6% of athletes responded that they had knowingly doped in the last 12 months.

Now, my initial reaction was one of skepticism in the methods of the survey. I'm still not sure I fully understand the study design (they used something called an Unrelated Question Model), but they go into some depth as to how they control for errors or unintended "yes" responses. Specifically, they looked at how the numbers change if they took out the top 10-50% of respondents in terms of how fast they took the survey - the thinking being that some athletes may have taken the survey so quickly they didn't actually stop to pay attention to what they were answering. "The prevalence estimates decreased upon deleting the 10%-20% of fastest respondents, but stabilized thereafter.

You can read the full text here:

Link:www.sportsintegrityinitiative.com/wp-c...

So assuming this survey is statistically sound and that no clean athlete would knowingly respond "yes"... why in the world would these athletes answer honestly? What's the incentive? The study was anonymous and randomized, but you don't dodge doping control by being honest. I'm still trying to wrap my head around that part.

But 43.6% is a lot. Especially when you consider that a significant portion of doping respondents were probably smart enough to not answer "yes." It's hard to not look at this study and think that the whole sport is dirty top to bottom.

rekrunner wrote:

Armstronglivs wrote:

You have just summoned rekrunner, as we see. He has been trying to discredit the survey - without success - for years here.

The main point, which is irrefutable, is that the incidence of doping at championship levels far exceeds the numbers caught.

Doping will certainly have continued to develop in the last decade, as much of it is based on advances in medicines and the rewards for doping remain a powerful incentive.

These were nobody's main points. No one in history has ever argued doping prevalence is as low as 1-2%. Their main point was to estimate the "true" doping prevalence.

My "success" is highly dependent on the reader. Independent of my success, the study speaks for itself, for those prepared to read all of it, and able to understand what the mathematical models represent. Their main point was to propose an anonymous survey as a useful tool to estimating "true" doping prevalence, while simultaneously identifying, but not resolving, no less than 8 barriers that would need to be overcome, standing between their survey results and true doping prevalence.

Much of the "debate" here over the years was about properly interpreting the study, whether the unrefined "43.6%", or post-processed "31.4%" after eliminating the fastest responses, was their "true" estimate, and whether the researchers rationalized that it was "likely more than" 43.6% or 31.4%.

But the rationalized opinions of the authors are not particularly relevant. Given all the unresolved unknowns identified in the study, there is no guarantee that any of the study results or opinions are "true" or even "likely". As we have seen, despite the often discussed 43.6% and 31.4% figures, the study itself presented no less than 39 potential estimates, in an attempt to eliminate potential distortions, ranging from 8.9% to 73.1%. Even these are not strict lower and upper bounds, but rather a result of an arbitrarily chosen 30% non-compliance. We have no way of identifying which of these 39 estimates are the closest, when determining the "true" prevalence requires resolving all of the unknowns, and passing the unfiltered results through a combination of all 8 mathematical filters.