Critical Regions

A rule fixed before the data

A hypothesis test starts from the null hypothesis, $H_0$H₀, which usually says that nothing has changed. If $H_0$H₀ is true, the test statistic z follows the standard normal distribution: values near 0 are common, and values far out in either direction are rare.

Before any data is collected, choose a significance level, often 5 percent. Then mark off the most extreme 5 percent of the distribution, on the side the alternative hypothesis $H_1$H₁ points to. Those values of z are the critical region, and the value where the region begins is the critical value.

For a test of whether a mean has increased, the region is the top 5 percent. $P\left(Z\ge 1.645\right)=0.05$P(Z ≥ 1.645) = 0.05, so the critical value is 1.645 and the critical region is $z\ge 1.645$z ≥ 1.645.

The gold curve is the standard normal distribution, the distribution of z if $H_0$H₀ is true. The shaded tail from 1.645 holds 5 percent of the area: that tail is the critical region.

Where the result lands

Now collect the data and work out z. Suppose it comes to 2.5. That lies beyond 1.645, inside the critical region. If $H_0$H₀ were true, a z of 2.5 or more would happen with probability $P\left(Z\ge 2.5\right)=0.0062$P(Z ≥ 2.5) = 0.0062, about 6 times in 1,000. A result that rare counts as evidence against $H_0$H₀, so reject $H_0$H₀ at the 5 percent level.

Suppose instead that z comes to 1.2. That is short of 1.645, outside the region. Values of 1.2 or more happen with probability $P\left(Z\ge 1.2\right)=0.1151$P(Z ≥ 1.2) = 0.1151 when $H_0$H₀ is true, more than one time in ten, so do not reject $H_0$H₀.

Not rejecting $H_0$H₀ does not prove it true. It means only that this data is not surprising enough to rule it out at the level chosen.

The same critical region, with two possible results marked. z = 2.5 lies under the shaded tail, so it rejects $H_0$H₀. z = 1.2 lies to the left of the shading, so it does not.

Which tail, and how far out

The alternative hypothesis decides where the region goes, and it must be chosen before the data is seen. For a test of whether a mean has decreased, the region is the lowest 5 percent: $z\le -1.645$z ≤ −1.645.

For a test of whether a mean has changed in either direction, a two-tailed test, the 5 percent is split into 2.5 percent in each tail. $P\left(Z\ge 1.96\right)=0.025$P(Z ≥ 1.96) = 0.025, so the region is $z\le -1.96$z ≤ −1.96 or $z\ge 1.96$z ≥ 1.96. The critical values sit further out than 1.645, because each tail holds only half as much.

A smaller significance level moves the critical values further out still. At 1 percent, a one-tailed test has critical value 2.326, since $P\left(Z\ge 2.326\right)=0.01$P(Z ≥ 2.326) = 0.01, and a two-tailed test has critical values $\pm 2.576$±2.576, since 0.005 lies beyond each of them.

It opens on a two-tailed test with z at 1.5: the two shaded tails hold about 13 percent, too much to reject $H_0$H₀. Drag z out to 1.96, where they hold 5 percent, 2.5 percent in each; switch to one tail and the dashed critical value moves in to 1.645.

The region in the units of the data

A critical region can be written for the sample mean itself instead of z. A machine fills boxes of cereal with a mean of 500 g and a standard deviation of 12 g. A manager suspects that it has started to overfill, so $H_0$H₀: $\mu =500$μ = 500 and $H_1$H₁: $\mu >500$μ > 500, at the 5 percent level, using a sample of 36 boxes.

The standard error is $12÷\sqrt{36}=12/6=2\text{g}$12 ÷ √36 = 12 / 6 = 2 g. The critical region is $z\ge 1.645$z ≥ 1.645, and $z=\left(\bar{x}-500\right)/2$z = (x̄ − 500) / 2, so the region is $\bar{x}\ge 500+1.645\times 2=503.29\text{g}$x̄ ≥ 500 + 1.645 × 2 = 503.29 g.

The 36 boxes have a mean of 503.8 g. That is in the critical region, so reject $H_0$H₀: there is evidence at the 5 percent level that the machine is overfilling. As a check, $z=\left(503.8-500\right)/2=1.9$z = (503.8 − 500) / 2 = 1.9, which is past 1.645.

When the result is a count

For a count, the critical region is a set of whole numbers, and its probability usually cannot be exactly 5 percent. A coin is tossed 10 times to test whether it favors heads, with $H_0$H₀: p = 0.5 and $H_1$H₁: p > 0.5. Under $H_0$H₀ the number of heads X follows B(10, 0.5).

Add the probabilities from the top. $P\left(X\ge 9\right)=\frac{10+1}{1024}=\frac{11}{1024}=0.0107$P(X ≥ 9) = (10 + 1)/1024 = 11/1024 = 0.0107, and $P\left(X\ge 8\right)=\frac{45+10+1}{1024}=\frac{56}{1024}=0.0547$P(X ≥ 8) = (45 + 10 + 1)/1024 = 56/1024 = 0.0547. The first is below 0.05 and the second is above it, so the critical region is $X\ge 9$X ≥ 9.

The probability of the region, 0.0107, is the actual significance level of the test: the chance of rejecting $H_0$H₀ when the coin is in fact fair. It is below the 5 percent asked for, because the next value down, 8, would take the region over 5 percent.

The usual mistakes

Choosing the region after seeing the data. Once the result is known, a region can always be drawn around it. The level, the tail and the critical value are all fixed first.

Using the wrong critical value. 1.645 belongs to a one-tailed test at 5 percent and 1.96 to a two-tailed test at 5 percent. A two-tailed test with 1.645 puts 5 percent in each tail, 10 percent in all.

Letting a count’s region go over the level. For the coin, $X\ge 8$X ≥ 8 has probability 0.0547, which is more than 0.05, so 8 is not in the region, however close 0.0547 is to 5 percent.

Saying that $H_0$H₀ is proved. A result outside the region fails to reject $H_0$H₀; it does not show that $H_0$H₀ is true.

A basketball player’s free throws

In the application below, a player who scored 40 percent of her free throws last season takes 20 throws after a summer of coaching. Under $H_0$H₀ her score follows B(20, 0.4), and the critical region is found from the top, the same way as for the coin.