A Fixed Roulette Wheel

In the comment section of this post, Bob Murphy asks how I would respond to a paper beginning:

"Abstract: It is well-known that players at the craps table are said to have a 'hot hand' after several advantageous rolls. The rollers themselves often report subjectively feeling 'in the zone' during streaks of successful rolls. However, using both Monte Carlo simulations and Bayesian inference models, we conclude that such 'patterns' are illusory and provide no operationally useful betting opportunity."

The idea is sound, but I think the point Bob wants made can be illustrated even better with an example from Willful Ignorance, a book which Ken B. recommended to me, but now seems to be willfully ignoring! (Sorry, Ken, I could not resist that joke.)

The author tells the story of George, a bright inventor who has figured out how to hack a casino's roulette wheel so that it produces a winning number he wants on command. So he could, say, produce one hundred 26s in a row, and clean up by continually betting on 26. But George is a lot smarter than that: he has seen the movies where people are beat up in the back room of the casino for doing that sort of thing. What he does instead is to grab a random number generator app for his phone, and have it randomly pick a number between 0 and 37 (with 37 representing 00), and then cause that number to "hit" on the wheel. (And of course he has several different accomplices win, rather than winning himself, and only on a few spins an evening.)

Clearly, this is no longer a "fair" roulette wheel, at least for George and his friends or for the casino. (It still is fair for the other players! Their chance of winning is unchanged by George's scheme.) On whatever occasions George decides to use his device, the outcome it is not due to "chance,"* but is being deliberately selected.

But no statistical test applied to the pattern of winning numbers will detect anything but chance at work. If Gilovich, Tversky and Vallone used the method of their famous hot hand paper on this wheel, they would have to conclude that George's idea that he could beat the wheel was just an illusion! (Of course, if researchers had more knowledge, specifically, the knowledge of who George's accomplices were, they could detect the scheme by analyzing those players' winning percentages.)

The point of the story is that there can be real causal factors at play in a situation that will not be revealed by the obvious statistical tests. A statistical test that concludes "No significant effect was found" should be a piece of evidence in the trial of a hypothesis, and not the verdict of the trial.

* A side note: "chance" is not properly speaking the cause of anything. At the quantum level, as Ken pointed out, we perhaps find truly random events. But that is just to say that it is possible that, for instance, an excited electron dropping back to a lower atomic orbital is a causeless event. It does not mean some pagan god called "Chance" made the electron shift orbits. And at the macro level, "chance" is just the name we give to a situation in which a myriad of causal factors are at play, and it is beyond our ken (b.) to sort them all out.