I wonder what standard of evidence would be appropriate. He says 0.3% of happening randomly. Another way of stating that is that his test has a 0.3% false positive rate. And given the number of shows produced, that ends up a high absolute number.
He analyzed one episode. The 0.3% probability is of that pattern happening in that episode. It would be different if he went searching for patterns through all episodes, but that is not what he did.
I think that is still an uncharitable interpretation of events. No one is claiming (so far as I know) that Who Wants to be a Millionaire? scoured all episodes looking for suspicious patterns. This is one episode where cheating was already suspected (though I would like to know on what grounds they suspected it).
>No one is claiming (so far as I know) that Who Wants to be a Millionaire? scoured all episodes looking for suspicious patterns
Because it's normally not worth it. This selected a show with a big win where it was worth analyzing it.
Big winners will always either know a lot, or be lucky (and bold) with their guesses. So this is already a pattern that was selected for.
Now given this backwards reasoning every time the guest said something else but changed his mind will look like cheating. But you already selected only the cases where they did get lucky, so that shouldn't factor in to your thinking, but it obviously does because humans are bad at separating this rationally.
No matter the percentages, absent any proof of collusion I don't think this conviction holds up, since nervous cough when seeing the right answer is enough of an explanation to make this plausible without any connection between the two.
You are arguing a different point than the original comment I was replying to, however. The original argument was that even with only a 0.3% probability of being coincidental, you would still expect to find one episode out of the entire run where it occurs. This is true. However, that doesn't make it likely that this episode is the one. If they had trawled the whole series looking for weirdness, and picked out the one episode with weirdness, that would be a major issue. They didn't do that. They picked this specific episode for a different reason (that I am still not completely clear on). The analysis shows that the coughing pattern was extremely unlikely to be coincidental. You are correct that doesn't necessarily mean cheating, but it is an interesting result that shouldn't simply be dismissed with hand wavy statistical arguments. The correct follow-up is as mentioned in the article. If the nervous coughing explanation is correct (for example) then most episodes should show a similar pattern which would undermine the conviction.
>If the nervous coughing explanation is correct (for example) then most episodes should show a similar pattern which would undermine the conviction.
That doesn't necessarily follow. It could still be that this one person has a nervous cough response to correct answers. Doesn't mean everyone with a cough does the same.
This neglects selection effects. There's a reason he's watching this episode instead of other episodes - if the studio investigates every large win and only makes noise about it when they have some level of evidence, as here, then there's a strong selection effect.
From Section 5 (Summary): "This analysis has some obvious limitations. ... Ideally, we would also analyse other episodes of WWTBAM and see if the distribution of coughs is similar to this infamous episode. (I hope to do this when I have a chance.)"
I'm looking forward to that analysis as I am sure you are too.
But coughs is not the only relevant variable. You'd have to analyze everything that could plausibly be used as an accusation of cheating. Looking only at coughs is already falling prey to selection bias.
not really, if they say coughs were the method of cheating then you can just look at coughs. if their claim was any noise counts,then yes you have to look at sneezes or any other sound. however to be able to use this then you need to know what you are listening for, so a range of sounds is not effective.
Suppose there's a hundred different possible sounds, and each sound has a 1% chance of randomly correlating . The probability of at least one sound correlating would be very high, but once that's identified you would calculate a 1% chance there.
That's selection bias. You selected the cough to look at because coughs were the ones that correlated. If something else correlated you might have looked at that one.
it may aid your understanding if you were. specifically this guy used up his lifelines in the easiest questions, and then had a clear run on the difficult ones. it was a questionable performance, hence there being an investigation and refusal to award the prize immediately.
This statistical analysis does not account for the fact that there is such a thing as 'nervous cough' which is anxiety-related. Stress hormones are likely to spike during stressful moments around the time a question is asked which may trigger a cough reflex from audience members.
The analysis seems to focus on the wrong things. A proof of cheating should focus more on identifying specific and reliable patterns in the coughs. I don't think generic statistical analysis of this kind makes a strong case at all.
the crime is to dishonestly acquire something, in this case the actual charge was fraud IIRC.
They conspired to win the prize in a way that was outside the rules, by obtaining help that they were not allowed to use. so the crime isn't winning it's how you won, and then did you really win or did you cheat.
much like in a sporting event, you can win by taking performance enhancing drugs, just because you came first doesn't mean you won.
Fraud is a tricky thing. It requires not only deceit, but 'legal injury'. The show producers clearly had great ratings, a popular show and considerable exposure due to the sensational nature of this family's win. Their winnings was a tiny fraction of a marketing budget. Arguably not a lot of legal injury there.
'But it was not fair!' is not a legal argument. Neither is 'outside the rules' since rules are not laws or regulations. Just some words the director spoke in their direction; hot air.
This. The sums of money at stake in soccer are far greater, the rules are much clearer and the participants routinely cheat yet no-one is ever prosecuted. Even the suggestion they ought to be seems risible.
> A person who dishonestly, with a view to gain for himself or another or with intent to cause loss to another, by any deception procures the execution of a valuable security shall on conviction on indictment be liable to imprisonment for a term not exceeding seven years; and this subsection shall apply in relation to the making, acceptance, indorsement, alteration, cancellation or destruction in whole or in part of a valuable security, and in relation to the signing or sealing of any paper or other material in order that it may be made or converted into, or used or dealt with as, a valuable security, as if that were the execution of a valuable security.
He intended to gain, and he collaborated with someone else to cheat. Loss isn't required.
Under the newer law (Fraud Act 2006) it's not necessary to show any gain.
> It is not necessary to prove or demonstrate any consequences of fraud (though they will clearly be material to sentence, compensation and confiscation). "Preddy" type difficulties will not arise (where the property obtained had not belonged to another);
To me it feels odd that you'd need to show a loss. How would you prosecute people who attempted but failed to defraud others?
But...'cheating' at a game, making that have the force of law, makes game-show hosts into lawmakers. That seems a strong stance. We're not talking a contract, nor a physical exchange of goods and services, nor anything else normally associated with civil cases. We're talking, the accused may have operated outside a set of arbitrary rules made by some civilian for fun.
Miss the point any? To be criminally prosecuted for not following a rule some producer made up, makes that rule into law. Because its enforceable in a court apparently. Really?
A p-value of 0.3% is not that low given the multiple hypothesis testing nature of the show.
A quick Google result shows that 592 total episodes have been filmed of the show. Under the null hypothesis that there was no cheating in any of the shows, you would still expect more likely than the chance that some show has a strong p-value of 0.3% for cheating to be 1-.997^592 = 83%.
As a jury, I don't think I would convict with just a p-value of 0.3% alone. You'd want something like a DNA test with a p-value expressed in 10^-6 or something to reduce the problem of multiple hypothesis testing / setup assumptions. Or you'd want to rely on other evidence like a confession or wiretap.
I haven't heard of that case before, but I found the evidence presented in the blogpost not very convincing. The first argument states that assuming that coughs are independent, the cough pattern is unlikely. But the data later shows that they are hardly independent but cluster, often two or more coughs are in a short timespan.
The second argument goes through each of the questions, but doesn't really show any evidence for cheating. Question 8 has no suspicious coughs (coughs close to the correct answer), and Ingram knows the answer quickly. Question 9 contains one suspicious cough, but only after the second mention of the correct answer. Again Ingram is quite sure about this answer and does not consider any other answer.
Question 10 is the first question (of that day) Ingram struggles with. There are 7 coughs recorded, 5 cough "clusters" (coughs very close to each other), 2 cough cluster could be suspicious. The first potentially suspicious cough cluster is pretty far away from the answers , while the second one is close to the correct answer. The next notes that these coughs didn't come from the phantom cougher, but from his wife. On the other hand there are lots of better place to insert a cough to cheat. This is one of the few questions where he changes his mind.
Question 11 has five coughs in total and three suspicious coughs, but cough 2-5 are very close together (and very late). Ingram is focused on the correct answer from the beginning, and does not really consider any other answer.
Question 12 has one cough after an incorrect answer at the very beginning. There is one cough cluster later on after the correct answer, but the correct answer is said 5 times, whereas the only one other answer is mentioned once, and only considered for a few seconds. The other coughes are relatively far away from any answer.
Question 13 has a cough after the correct answer in the very beginning, but it is not called as a significant cough in the linked youtube video from WWTBAM. Only the correct answer is considered.
In Question 14 Ingram struggles similar to question 10. there are 12 coughs. 6 coughs are heard shortly after a wrong answer (coughs 1, 2, 3, 4, 5 and 9), and 4 coughs are heard shortly after the right answer (coughs 6, 10, 11, 12). The next mentions a muttered "no", but any muttering loud enough to inform the contestant surely must be heard by the host and the crew. If they cheated using coughs, why muttering very obviously?
Question 15 has 23 coughs, 7 are close to wrong answers and 10 are close to the right answer. Ingram speaks the correct answer 11 times, and lists the incorrect answers 4 times. Most of the suspicious coughing happens in the very end, when he basically made his decision. Most of the coughing close to the incorrect answers happens early. If coughing was used, why did he not lock into one of the incorrect answers? The text mentions Ingram muttering "I think I know..." and calls that suspicious, but that phrase seems a very natural choice here.
The questions, coughing and answering doesn't follow a pattern: Sometimes there is a cough very early after the correct answer and Ingram picks that option, sometimes there is a cough very early after the incorrect answer put Ingram doesn't pick that option. Sometimes for unclear questions there are multiple coughs after the correct option, and Ingram picks that option; sometimes for unlcear options there are multiple coughs after the incorrect option and Ingram does not pick that option. On some unclear questions there is no coughing at all, but Ingram still guesses correctly. On some clear questions there is coughing, although Ingram did not even consider any wrong answer.
Next the blog post compares distributions of elapsed time since last answer for correct and incorrect answers. It does not use a statistical test (e.g. Kolmogorow-Smirnow-Test), but just counts coughs to some threshold. Resolution is very rough, the subsampling for that plot only done once. While simulations show ...
33 comments
[ 2.3 ms ] story [ 88.2 ms ] threadBecause it's normally not worth it. This selected a show with a big win where it was worth analyzing it.
Big winners will always either know a lot, or be lucky (and bold) with their guesses. So this is already a pattern that was selected for.
Now given this backwards reasoning every time the guest said something else but changed his mind will look like cheating. But you already selected only the cases where they did get lucky, so that shouldn't factor in to your thinking, but it obviously does because humans are bad at separating this rationally.
No matter the percentages, absent any proof of collusion I don't think this conviction holds up, since nervous cough when seeing the right answer is enough of an explanation to make this plausible without any connection between the two.
That doesn't necessarily follow. It could still be that this one person has a nervous cough response to correct answers. Doesn't mean everyone with a cough does the same.
I'm looking forward to that analysis as I am sure you are too.
Suppose there's a hundred different possible sounds, and each sound has a 1% chance of randomly correlating . The probability of at least one sound correlating would be very high, but once that's identified you would calculate a 1% chance there.
That's selection bias. You selected the cough to look at because coughs were the ones that correlated. If something else correlated you might have looked at that one.
The analysis seems to focus on the wrong things. A proof of cheating should focus more on identifying specific and reliable patterns in the coughs. I don't think generic statistical analysis of this kind makes a strong case at all.
And what's the big deal? Is it actually a crime to win a game show? I'm confused.
They conspired to win the prize in a way that was outside the rules, by obtaining help that they were not allowed to use. so the crime isn't winning it's how you won, and then did you really win or did you cheat.
much like in a sporting event, you can win by taking performance enhancing drugs, just because you came first doesn't mean you won.
'But it was not fair!' is not a legal argument. Neither is 'outside the rules' since rules are not laws or regulations. Just some words the director spoke in their direction; hot air.
I never understood the point of this prosecution.
> A person who dishonestly, with a view to gain for himself or another or with intent to cause loss to another, by any deception procures the execution of a valuable security shall on conviction on indictment be liable to imprisonment for a term not exceeding seven years; and this subsection shall apply in relation to the making, acceptance, indorsement, alteration, cancellation or destruction in whole or in part of a valuable security, and in relation to the signing or sealing of any paper or other material in order that it may be made or converted into, or used or dealt with as, a valuable security, as if that were the execution of a valuable security.
He intended to gain, and he collaborated with someone else to cheat. Loss isn't required.
Under the newer law (Fraud Act 2006) it's not necessary to show any gain.
https://www.cps.gov.uk/legal-guidance/fraud-act-2006
> It is not necessary to prove or demonstrate any consequences of fraud (though they will clearly be material to sentence, compensation and confiscation). "Preddy" type difficulties will not arise (where the property obtained had not belonged to another);
To me it feels odd that you'd need to show a loss. How would you prosecute people who attempted but failed to defraud others?
How? They're not adding anything into law.
> nor anything else normally associated with civil cases
This is a criminal, not civil, case.
Being a criminal case makes it more egregious.
[0] https://en.wikipedia.org/wiki/Deception_(criminal_law)
A quick Google result shows that 592 total episodes have been filmed of the show. Under the null hypothesis that there was no cheating in any of the shows, you would still expect more likely than the chance that some show has a strong p-value of 0.3% for cheating to be 1-.997^592 = 83%.
As a jury, I don't think I would convict with just a p-value of 0.3% alone. You'd want something like a DNA test with a p-value expressed in 10^-6 or something to reduce the problem of multiple hypothesis testing / setup assumptions. Or you'd want to rely on other evidence like a confession or wiretap.
If it was found by searching all of 23andMe it would be meaningless on its own.
The second argument goes through each of the questions, but doesn't really show any evidence for cheating. Question 8 has no suspicious coughs (coughs close to the correct answer), and Ingram knows the answer quickly. Question 9 contains one suspicious cough, but only after the second mention of the correct answer. Again Ingram is quite sure about this answer and does not consider any other answer.
Question 10 is the first question (of that day) Ingram struggles with. There are 7 coughs recorded, 5 cough "clusters" (coughs very close to each other), 2 cough cluster could be suspicious. The first potentially suspicious cough cluster is pretty far away from the answers , while the second one is close to the correct answer. The next notes that these coughs didn't come from the phantom cougher, but from his wife. On the other hand there are lots of better place to insert a cough to cheat. This is one of the few questions where he changes his mind.
Question 11 has five coughs in total and three suspicious coughs, but cough 2-5 are very close together (and very late). Ingram is focused on the correct answer from the beginning, and does not really consider any other answer.
Question 12 has one cough after an incorrect answer at the very beginning. There is one cough cluster later on after the correct answer, but the correct answer is said 5 times, whereas the only one other answer is mentioned once, and only considered for a few seconds. The other coughes are relatively far away from any answer.
Question 13 has a cough after the correct answer in the very beginning, but it is not called as a significant cough in the linked youtube video from WWTBAM. Only the correct answer is considered.
In Question 14 Ingram struggles similar to question 10. there are 12 coughs. 6 coughs are heard shortly after a wrong answer (coughs 1, 2, 3, 4, 5 and 9), and 4 coughs are heard shortly after the right answer (coughs 6, 10, 11, 12). The next mentions a muttered "no", but any muttering loud enough to inform the contestant surely must be heard by the host and the crew. If they cheated using coughs, why muttering very obviously?
Question 15 has 23 coughs, 7 are close to wrong answers and 10 are close to the right answer. Ingram speaks the correct answer 11 times, and lists the incorrect answers 4 times. Most of the suspicious coughing happens in the very end, when he basically made his decision. Most of the coughing close to the incorrect answers happens early. If coughing was used, why did he not lock into one of the incorrect answers? The text mentions Ingram muttering "I think I know..." and calls that suspicious, but that phrase seems a very natural choice here.
The questions, coughing and answering doesn't follow a pattern: Sometimes there is a cough very early after the correct answer and Ingram picks that option, sometimes there is a cough very early after the incorrect answer put Ingram doesn't pick that option. Sometimes for unclear questions there are multiple coughs after the correct option, and Ingram picks that option; sometimes for unlcear options there are multiple coughs after the incorrect option and Ingram does not pick that option. On some unclear questions there is no coughing at all, but Ingram still guesses correctly. On some clear questions there is coughing, although Ingram did not even consider any wrong answer.
Next the blog post compares distributions of elapsed time since last answer for correct and incorrect answers. It does not use a statistical test (e.g. Kolmogorow-Smirnow-Test), but just counts coughs to some threshold. Resolution is very rough, the subsampling for that plot only done once. While simulations show ...