Lady tasting tea

In the design of experiments in statistics, the lady tasting tea is a randomized experiment devised by Ronald Fisher and reported in his book The Design of Experiments (1935). The experiment is the original exposition of Fisher's notion of a null hypothesis, which is "never proved or established, but is possibly disproved, in the course of experimentation".

The lady in question (Dr. Muriel Bristol) claimed to be able to tell whether the tea or the milk was added first to a cup. Fisher proposed to give her eight cups, four of each variety, in random order. One could then ask what the probability was for her getting the specific number of cups she identified correct, but just by chance.

Fisher's description is less than 10 pages in length and is notable for its simplicity and completeness regarding terminology, calculations and design of the experiment. The example is loosely based on an event in Fisher's life. The test used was Fisher's exact test.

The experiment provided the Lady with 8 randomly ordered cups of tea—4 prepared by first adding milk, 4 prepared by first adding the tea. She was to select the 4 cups prepared by one method. This offered the Lady the advantage of judging cups by comparison. She was fully informed of the experimental method.

The null hypothesis was that the Lady had no ability to distinguish the teas. In Fisher's approach, there is no alternative hypothesis; this is instead a feature of the Neyman–Pearson approach.

The test statistic was a simple count of the number of successes in selecting the 4 cups. The null hypothesis distribution was computed by the number of permutations. The number of selected permutations equalled the number of unselected permutations. Using a combination formula, with ${\displaystyle n=8}$ total cups and ${\displaystyle k=4}$ cups chosen, there are ${\displaystyle {\frac {8!}{4!(8-4)!}}=70}$ possible combinations.

...
Wikipedia