An Examination Of The Timing/Scoring Screw-Ups At The 2012 NCAA Cross-Country Championships
Was This An Isolated, One-Off Mistake? Or Could It Have Been Prevented?
By LetsRun.com
December 4, 2012
Those of you that are fans of women’s cross-country undoubtedly know there was a big mistake made this year at the NCAA D1 Cross-Country Championships. Throughout the race, unofficial scores using the runners’ chip times were shown on the scoreboard and they showed the Oregon Ducks to be in control of the competition. After the race was over, unofficial chip scores showed Oregon as the winner.
Then the official results were announced and – drum roll, please – Providence was declared the official winner and #1-ranked Florida State wasn’t on the podium. After a protest, at least two missing runners – one for Oregon and one for FSU were found – and Oregon was re-declared to be the winner and FSU moved up to #4.
We decided to look into what at happened. We received an email from a coach of a top 25 NCAA women’s team who saw the chaos unfold first hand. He or she would like to remain anonymous, as criticizing the NCAA isn’t going to help his or her career prospects. You can read his or her’s account of the scoring chaos that happened at NCAAs here:
We also decided to try to figure out what happened from an administrative/technological/timing standpoint. What exactly happened? Was this screw-up an isolated incident? Could it have been prevented?
Fool Me Once, Fool Me Twice, Fool Me Three Times??
The first thing we realized is that this wasn’t an isolated incident. The timing company handling the D1 championship – Delta Timing – also handled the DIII championship this year as well as the D1 South Regional championships. All three championships were marred by timing/reporting issues.
Delta Timing reported that North Central won with 349 points. In reality, they won with 167 points. Photo from Mark Dannenhauer Photography.
Given its lower profile, many may not realize, but the DIII men’s championship was marred by scoring problems as well. At the DIII champs, the scores reported on the scoreboard apparently listed scores for teams if all seven runners, not just the top 5, were scored. The picture on the left shows they were astronomically high.
Also the individual results that were put up on the scoreboard initially weren’t 100% correct and some individuals incorrectly thought they were All-Americans when they weren’t (and we’re assuming the vice versa is true as well). Even as late as two weeks after the race was over, Wisconsin Eau-Claire actually had eight finishers listed in the race (apparently one guy wore a chip of a teammate who didn’t race).
At the women’s South Regional, Florida was incorrectly reported as being the #2 qualifier for NCAAs, when in reality Florida was fourth.
In investigating what happened with the various timing/scoring mishaps, we reached out to two people who both work in very high positions in the timing world. They both agreed to give LetsRun.com some scoop as long as they remained anonymous.
One of them tried to be sympathetic to the people at Delta Timing. He said they had been around a long time and probably were in the top five in the US in terms of total competitions timed. Click here to see background info on Delta, which has been around since 1993 and is celebrating its 20th anniversary this year.
“Technology lets you down and I don’t want to throw stones. I know shit happens,” he said.
We then asked him how he could match that sentiment with the fact that at the D1 champs, the technology seemed to have the scores close to right before a human seemingly messed it up (more on that later) and with the fact Delta had also had problems at two other NCAA meets. He responded as follows:
“Clearly something went wrong with the results process at the D1 and D3 champs. I wasn’t involved with the results production at any of the meets you have mentioned, so I have no idea what actually happened. That much being said, if there is a recurrent pattern of technology failure, then there are a lot of people in the timing industry who would like to know what equipment failed and how it failed. This is an industry where people share their experiences, their expertise, and their knowledge – for the good of the sport.”
The other timing person, who works at a rival timing company, was much less sympathetic. When we emailed him asking for his take on the situation, he wrote back (his full email appears at the end of this article) and stated based on Delta’s past practices, many in the timing world were virtually certain there was going to be a problem at the NCAA D1 champs, as he wrote:
“When I heard Delta was going to time the championships back in August, I knew there was going to be problems. In fact, I was talking to *Name Removed* on Monday morning Nov. 12, because he was asking me what I knew about the problems Delta had at the Regionals. I didn’t know anything about those issues, but I said something to *Name Removed* like ‘Thank God I’m going to be *Number Removed* miles away from Louisville this weekend because it’s going to be a mess.'”
Once we got this email, we wrote back and asked him, “How did you know it was going to be a problem? Do they (Delta) have a history of making mistakes or was it only because you heard they messed up the regionals?”
“Myself and many others in the business thought this would happen because Delta doesn’t have a real good reputation. They have been sloppy with track meets in the past (they did the USATF indoor meets (Millrose, etc.) for just one season back in 2000 when USATF had a contract dispute with Flash Results and it didn’t go so well). You can certainly ask around and I think you would hear the same thing.”
So What Actually Happened/Caused The Problem At The NCAA D1 Meet?
As for what actually happened at the D1 meet, we tried to hear from the horse’s mouth directly and reached out to Delta Timing, but they didn’t answer our questions directly. They referred us to Cameron Schuh at the NCAA, who emailed us the following statement:
“The NCAA has used Delta Timing as the official timing partner for the Division III cross country championship for years and we felt they were best qualified to handle the Division I championship this year. We continue to be pleased with the partnership.
During the 2012 Division I Women’s and Division III Men’s Cross Country Championships, there were two instances of technical glitches in the collection of the timing. This did lead to some confusion at the immediate conclusion of each race as misinformation was announced. Fortunately though, the championships are prepared for such instances and there are a number of back-up measures in place, including video cameras recording the order of finishers.
The NCAA and Delta Timing staff discovered the discrepancy and used the back-up measures to correctly determine the order of finishers, times and team champions. Unfortunately, as the review was taking place, the PA announcer made an announcement based on the info he had at that time, but it is widely known in the cross country community that race results aren’t official until 30 minutes after the second races. This is exactly what took place last week.
Once the race results were confirmed, the official announcement was made on-site and for the online viewers.
We do feel badly for those student-athletes and coaches who were negatively impacted by the misinformation during the period of time the review was taking place.
In the end however, the accurate individuals and teams were properly awarded the finishes they achieved, and in a timely enough manner that did not ruin the overall championship experience.”
We’re glad that the NCAA got back to us and agree that neither glitch ruined the meets, but their statement seemingly ignores the fact that the results showing Providence as the winner were indeed labelled “Official” (scroll down to see a photo of results with Providence listed as the champ and the word “Official” next to them).
The statement is also factually inaccurate. Delta has worked some DIII national track meets of late (they timed last year’s indoor and outdoor meets as well as the 2011 outdoor meet) but none in cross-country. As one long-time DIII coach told us, “For anyone to say that Delta timing has been timing the Division III cross country meet for a long time – that’s not true at all.” Our source at the rival timing company said he could see “no record of them timing DIII (cross country) prior to this year – Delta timed DIIs a couple of years ago at Louisville.” LetsRun looked at the last three years of results and confirmed that none of them were done by Delta.
But What Actually Happened At The D1 Meet?
The two timing executives that we talked to didn’t know for sure what happened at the D1 meet, but both thought it was a mistake that Delta and the NCAA seemingly tried to rely on chip scores for actual finish line scores. They both thought actual finish line chip scores, even if unofficial, shouldn’t ever be released.
Both executives said a timing company at a big meet like NCAAs has two jobs. One is to provide chip scores for entertainment purposes during the race. Fans/TV announcers get a rough idea of what is happening but everyone knows they are unofficial and subject to review.
The second is to provide official results which have to be manually reviewed to insure they are 100% accurate. These results simply have to be right and something you’d “bet your mortgage on.” At NCAAs, lots of people are finishing very close to each other, so there is no way for a chip to get the order of finish totally correct – someone must look at the photo finish.
That being said, LetsRun pointed out to both men that the chip scores provided after the finish at least had the NCAA Champ right – Oregon. Only later, when seemingly the timing people were manually doing the scores, did a runner for Oregon, Alexi Pappas (#3 in team scoring), and a runner for Florida State, Amanda Winslow (#10 in team scoring), get totally missed in the results. Thus in our minds, it seems like a human error likely caused the real screw-up – unless the computer reversed course on itself.
|
Initial Score On Telecast |
Updated Score On Telecast |
|
First Official Results Which Were Wrong |
Actual Final Results |
We’ve done the math for the eventual top 4 podium teams and the third photo above is the actual scores of the teams assuming that Pappas and Winslow were just missing – except for the fact that we think Providence should have had 175 and not 174, as their top two runners (who were both ahead of Winslow but behind Pappas) would score one less point each and their final three runners would score two points less each, for a net of eight less points than their actual total of 183. So we don’t know why it says 174 and not 175, but one point isn’t all that significant of a screw up when the margin of victory is nearly 70 points.
Conclusion
Timing is a very complex business. Getting everything 100% right in a timely fashion isn’t easy. An occasional mistake might be understandable every now and then, but a mistake at three straight meets – including one that declared the wrong national champion – is simply unacceptable. As a result, moving forward, we feel the NCAA should re-examine its relationship with Delta, particularly in cross-country.
But when we started working on this piece, the goal was never to discredit a timing company. Our goal was to try to figure out what happened and make sure that it doesn’t happen again. As a result, we think it’s important that Delta tell the running world what happened at this year’s meets, particularly the D1 champs (if they offer a mea cupla and offer to do next year’s meet for free, maybe they could be forgiven). We also think it’s important that the NCAA develop some sort of protocol as to how results are collected/released in the future. We think using chip times for splits is critical for the enjoyment of fans/viewers and our fear is that the NCAA is going to over-react and results will be painfully slow in coming out in 2013. In the past, many, including LetsRun.com co-founder Robert Johnson, have criticized the NCAA for taking so long to get the results out.
Despite what the timing experts say about using chip times at the finish, LetsRun.com believes they are critical for the fans’ enjoyment. The colossal mistake at the 2012 D1 champs of the wrong national champion being officially announced wasn’t caused by chip times. Chip times are always supposed to be unofficial and Providence was declared as the official winner.
Additionally, the screen shots of the unofficial top 10 and top 20 results that we took just minutes after the race was over – which must have come from chip times – were very accurate. There really only were two mistakes (and one of them was acceptable in our minds) as you’ll see below. Florida State, which really was beaten by Stanford by just 4 points, was placed ahead of them in third instead of fourth (an acceptable error). And Arizona, which really placed sixth, was somehow listed as having finished 11th (unacceptable error).
Unofficial Result Shown Online Using Chip Time/Actual Result/Difference
1. Oregon – 1st – correct
2. Providence – 2nd – correct
3. FSU – 4th – off by one
4. Stanford – 3rd – off by one
5. Michigan – 5th – correct
6. Duke – 7th – off by one
7. Uconn – 8th – off by one
8. Washington – 9th – off by one
9. New Mexico – 10th – off by one
10. Iowa State – 11th – off by one
11. Arizona – 6th – off by 5
12. Georgetown – 12th – correct
13. Cornell – 13th – correct
14. Penn State – 14th – correct
15. Notre Dame – 15th – correct
16. Michigan State – 16th – correct
17. Weber – 17th – correct
18. Arkansas – 18th – correct
19. BC – 19th – correct
20. Villanova – 20th – correct
So instead of blowing up the whole system, the NCAA should find out what happened between the chip times coming in and then the official results going out without Pappas or Winslow in them. And they should find out what happened with Arizona being so far off its unofficial result. Did one runner not wear a chip or did her chip not register? If you take our their #4th runner and replaced them with #6, it adds roughly 100 points to their score which moves them from #6 to #11.
Looking ahead, it’s critical that the NCAA hires someone who is really, really good when it comes to timing the NCAA cross-country championships. If a mistake is made – it easily can have disastrous implications because some administrator or announcer from the NCAA or meet host, who might only work cross-country/track and field part of the year, isn’t likely to be able to correct any problems on the fly if they occur (admittedly, we don’t know who from the NCAA was involved). An insider who follows the sport day in and day out might have said, “Wait a minute, let’s don’t publish these. Oregon was way ahead all race – what happened?” Clearly that didn’t happen and as a result, the NCAA needs to make sure they have someone who can get the results 100% right next year, in a timely fashion.
As one of the timing execs said to us via email:
“The bottom line was that this whole confirmation process used to occur, and was strictly controlled by only about 2 or 3 people that knew exactly how everything worked. Those people were not at the championships this year. In the past, there have only ever been those 2 or 3 people at the meet that completely understood the entire process and what the proper protocol should be … and those would most likely not be wearing an NCAA official jacket.”
The full email from our source at the rival timing company appears below:
Emailed Thoughts From A Timing Executive On The 2012 NCAA Scoring SNAFU
In this case, I really don’t know what happened from a technical aspect – but that’s besides point. Delta clearly has been using some “quick score” software this year at those regionals and championships that produces a quick score immediately at the finish based on the raw chip data. In my opinion, this is should not be done for finishline data EVER and most definitely not at that meet. Its fine to put up quick scores from the split points, but at the finishline the data must be verified before it is released. And yes, verification will take extra time, and its not instant, but at least when the results are posted, they are correct. It sounds like Delta released quick scores immediately and then as they verified the finish order against the Lynx photo, the scores were adjusted “live” and they must have made mistakes. Posting the scores and then adjusting them “live” in front of the crowd might seem like a cool idea, but its not, because people get confused as to what is happening. And as soon as any scores are posted- even labeled “Unofficial” – people will believe they are official.
The proper protocol established many years ago (at least since 2004) for the NCAA D1 Championships (and other championships) was to start reviewing finishline chip data against the photofinish picture immediately. As the first 10 or 15 individual positions were confirmed, those were shown on the scoreboard and announced. Then, as further finish places were reviewed and confirmed, those were shown on the scoreboard. No team scores were ever released while the photofinish review was going on. In the case of those years where there was live TV, when they needed to know the winning team, once the photofinish review had confirmed past the point in the results where the top 10 teams had placed all of their top 5 finishers, it was safe to let the TV crew and the winning team know so that they could start interviews and take footage with the trophy. Keep in mind, though, that this was not allowed if the point spread was close (with 10-20 points.) If that was the case, the photofinish review was completed before anyone announced a winner. Once all of the places in the race had been reviewed and confirmed, then the team scores were released and announced.
The bottom line was that this whole confirmation process used to occur, and was strictly controlled by only about 2 or 3 people that knew exactly how everything worked. Those people were not at the championships this year. In the past, there have only ever been those 2 or 3 people at the meet that completely understood the entire process and what the proper protocol should be … and those would most likely not be wearing an NCAA official jacket.
When I heard Delta was going to time the championships back in August, I knew there was going to be problems. In fact, I was talking to *Name Removed* on Monday morning Nov. 12, because he was asking me what I knew about the problems Delta had at the Regionals. I didn’t know anything about those issues, but I said something to *Name Removed* like, “Thank God I’m going to be *Distance Removed* miles away from Louisville this weekend because its going to be a mess.”
The situation is completely unfortunate because this technology can really help this sport, but when used incorrectly its disastrous.
After receiving this, LetsRun.com emailed back asking for a bit more clarification: One more key question here – when you heard they were doing it – why and how did you know it was going to be a problem? Do they have a history of making mistakes or was it only because you heard they messed up the regionals?
And myself and many others in the business thought this would happen because Delta doesn’t have a real good reputation. They have been sloppy with track meets in the past (they did the USATF indoor meets (Millrose, etc.) for just one season back in 2000 when USATF had a contract dispute with Flash Results and it didn’t go so well). You can certainly ask around and I think you would hear the same thing.
Do you have comments or insight into the timing world? Email us or post them on the messageboard. MB: Post Your Comments on our Examination Of The Timing/Scoring Screw-Ups At The 2012 NCAA Cross-Country Championships Here
More: *LRC An Insider Coach’s Account Of The “Colossal” Scoring Screw-Up At The 2012 NCAA Cross-Country Championships
*So WTF happened – what was the offical word on the Women’s race?