Confidence Intervals

One number and its error

A machine fills bags of flour, and the standard deviation of a bag’s mass is known to be 10 g. To estimate the mean mass $\mu$μ of all its bags, 100 bags are weighed, and their mean is x̄ = 502 g.

502 g is a point estimate of $\mu$μ: a single number. Another 100 bags would give a slightly different mean, and the point estimate on its own says nothing about how far from $\mu$μ it might be.

The standard error measures how far sample means typically fall from $\mu$μ. For samples of 100 it is $\sigma ÷\sqrt{n}=10÷\sqrt{100}=10/10=1\text{g}$σ ÷ √n = 10 ÷ √100 = 10 / 10 = 1 g.

Plus or minus 1.96 standard errors

Sample means are normally distributed about $\mu$μ, with standard deviation equal to the standard error. In the standard normal distribution, $P\left(Z\ge 1.96\right)=0.025$P(Z ≥ 1.96) = 0.025, so 2.5 percent lies beyond 1.96 in each tail and P(−1.96 < Z < 1.96) = 0.95.

So in 95 percent of samples, x̄ lands within 1.96 standard errors of $\mu$μ. When it does, $\mu$μ is within 1.96 standard errors of x̄. That gives the 95 percent confidence interval: $\bar{x}\pm 1.96\times$x̄ ± 1.96 × standard error.

For the flour, $502\pm 1.96\times 1=502\pm 1.96$502 ± 1.96 × 1 = 502 ± 1.96, which runs from 502 − 1.96 = 500.04 g to 502 + 1.96 = 503.96 g.

The gold curve is the standard normal distribution. The shaded middle, from −1.96 to 1.96, holds 95 percent of the area, leaving 2.5 percent in each tail.

What the 95 percent means

$\mu$μ is a fixed number, and the interval from 500.04 g to 503.96 g either contains it or does not. The 95 percent belongs to the method: if many samples of 100 bags were taken and an interval made from each, about 95 in every 100 of those intervals would contain $\mu$μ.

The interval is about the mean of all the bags. It does not say that 95 percent of bags weigh between 500.04 g and 503.96 g. Single bags vary with a standard deviation of 10 g, far more than that.

Other levels of confidence

A different level uses a different z. For 90 percent, 5 percent is left in each tail, and $P\left(Z\ge 1.645\right)=0.05$P(Z ≥ 1.645) = 0.05, so the interval is $502\pm 1.645$502 ± 1.645, from 500.355 g to 503.645 g.

For 99 percent, 0.5 percent is left in each tail, and $P\left(Z\ge 2.576\right)=0.005$P(Z ≥ 2.576) = 0.005, so the interval is $502\pm 2.576$502 ± 2.576, from 499.424 g to 504.576 g.

More confidence needs a wider interval. The 99 percent interval is 2.576 ÷ 1.645, about 1.57 times as wide as the 90 percent one.

A larger sample, a narrower interval

The standard error divides by $\sqrt{n}$√n. With 400 bags instead of 100, it is $10÷\sqrt{400}=10/20=0.5\text{g}$10 ÷ √400 = 10 / 20 = 0.5 g, and if the mean were again 502 g the 95 percent interval would be $502\pm 1.96\times 0.5=502\pm 0.98$502 ± 1.96 × 0.5 = 502 ± 0.98, from 501.02 g to 502.98 g.

Four times as many bags halves the width, because $\sqrt{4}=2$√4 = 2. To halve the width again would take 1,600 bags.

Intervals and tests

Suppose the bags are labeled 500 g. That value lies below the 95 percent interval, 500.04 g to 503.96 g, so a mean of 500 g is not a plausible value at this level.

A two-tailed test of $H_0$H₀: $\mu =500$μ = 500 at the 5 percent level reaches the same verdict: $z=\left(502-500\right)/1=2$z = (502 − 500) / 1 = 2, which is beyond 1.96, so $H_0$H₀ is rejected. A claimed mean outside the 95 percent interval is always rejected by this test, and one inside it never is.

The usual mistakes

Going one standard error either side. $502\pm 1$502 ± 1 contains $\mu$μ only about 68 times in 100. A 95 percent interval needs 1.96 standard errors.

Doubling the margin. The $\pm$± already reaches both ways, so the half-width is 1.96 × 1 = 1.96, not 3.92.

Using $\sigma$σ instead of the standard error. $502\pm 1.96\times 10$502 ± 1.96 × 10 is a range for single bags, not for the mean.

Saying there is a 95 percent chance that $\mu$μ is in this interval. The 95 percent describes how often the method succeeds, not this one interval.

Journey times on a bus route

In the application below, 64 journeys are timed and $\sigma$σ is estimated from the sample as 8 minutes, so the standard error is $8÷\sqrt{64}=1$8 ÷ √64 = 1 minute. The timetable’s 32 minutes is then set against the interval.