How to test without making mistakes

In a paper published in 1931, statisticians Jerzy Neyman (1894–1981) and Egon Sharpe Pearson (1895–1980) laid the foundations for the study of hypothesis testing, introducing a formalism that is still used today, although it is increasingly criticized. In inferential statistics, a hypothesis test is used to make a decision about an (unknown) population parameter on the basis of a (known) estimate of that parameter from a random sample of the population (a classic example of such a test is given in the box).

To perform a statistical hypothesis test, we first formulate a basic (conservative) hypothesis about the population, called H₀, together with an alternative hypothesis H₁. We then draw a random sample from the population and calculate the statistic of interest for that sample. A decision rule then tells us whether the statistic observed in the sample is compatible with the hypothesis H₀ about the population—or, more precisely, the extent to which the observed statistic is compatible with H₀. An a priori probability calculation determines whether the observed result is likely or unlikely, given the hypothesis H₀.

Possible errors ----------------------

A hypothesis test can produce four possible outcomes, depending on the true situation and the test's conclusion.