The Central Limit Theorem

A lopsided population

Ask the students of a school how many library books they borrowed last month. Suppose 40% borrowed none, 30% borrowed 1, 15% borrowed 2, 10% borrowed 3 and 5% borrowed 4.

This distribution is nothing like a bell. It is highest at 0 and trails off to the right: it is skewed to the right. Its mean is $\mu =0\times 0.4+1\times 0.3+2\times 0.15+3\times 0.1+4\times 0.05=1.1$μ = 0 × 0.4 + 1 × 0.3 + 2 × 0.15 + 3 × 0.1 + 4 × 0.05 = 1.1 books, and its variance is ${\sigma }^2=1.39$σ² = 1.39, so $\sigma \approx 1.18$σ ≈ 1.18.

The population, as percentages of students: 40% borrowed no books, 30% one, 15% two, 10% three and 5% four. The tallest bar is at 0, and the bars fall away to the right.

Means of two

Choose 2 students at random and take the mean number of books they borrowed. It can be 0, 0.5, 1, and so on up to 4. The probability that both borrowed none is 0.4 × 0.4 = 0.16; the mean is 0.5 when one borrowed none and the other one, which has probability 2 × 0.4 × 0.3 = 0.24.

The distribution of the mean of 2 is still skewed to the right, but less so than the population: the mean of 2 is 0 only 16% of the time, against 40% for one student, because both students now have to borrow nothing.

The mean of 2 students, as percentages: highest at 0.5, and still trailing off to the right.

Means of ten

Now take the mean of 10 students. Its distribution, worked out exactly from the population, is almost symmetric. It peaks at 1.0 and 1.1, right at $\mu =1.1$μ = 1.1, and falls away at about the same rate on both sides: the chance of a mean of 0.5 or less is 0.061, and the chance of a mean of 1.7 or more, the same distance above 1.1, is 0.077.

The center has not moved: the mean of the distribution is still 1.1. Its variance is 1.39 ÷ 10 = 0.139, as Var$\left(\bar{X}\right)=\frac{{\sigma }^2}{n}$(X̄) = σ²/n says. What has changed is the shape. The lopsided population has produced sample means that pile up in a near-symmetric hump.

The mean of 10 students, as percentages, in steps of 0.1 from 0 to 2.5. The hump is nearly symmetric about 1.1, with a slightly longer tail on the right.

The central limit theorem

This is what the central limit theorem says. Take random samples of size n from any population with mean $\mu$μ and variance ${\sigma }^2$σ². When n is large, the sample mean X̄ is approximately normally distributed, with mean $\mu$μ and variance $\frac{{\sigma }^2}{n}$σ²/n. That is, X̄ is approximately $N\left(\mu ,\frac{{\sigma }^2}{n}\right)$N(μ, σ²/n), and the larger n is, the better the approximation. It holds whatever the shape of the population.

The theorem is about the means of samples, not about the population. The population of borrowings is just as skewed after any number of samples are taken; it is the distribution of the sample mean that becomes normal.

How large is large? About 30 is usually enough. A population that is already symmetric needs fewer, and one that is badly skewed needs more. If the population is itself normal, X̄ is exactly normal for every n.

Six hundred sample means from a population with mean 1 and standard deviation 1 that is strongly skewed: most values are small and a few are large. Drag n from 16 up to 100, and the pile of means grows narrower and more symmetric, a bell centered on 1.

Why the normal curve is everywhere

Many quantities are the sum or the mean of a large number of small, independent effects. A person’s height depends on many genes and on years of food and health; an error in a measurement is the sum of many small disturbances. The central limit theorem says that such sums and means come out close to normal, whatever each small effect looks like on its own. That is why so many measurements follow a normal curve.

Using it

Take a random sample of 30 students. By the central limit theorem, their mean number of borrowings is approximately normal, with mean 1.1 and standard error $\sqrt{1.39÷30}\approx 0.2153$√(1.39 ÷ 30) ≈ 0.2153.

To find the probability that the sample mean is more than 1.5, standardize: $z=\left(1.5-1.1\right)/0.2153\approx 1.858$z = (1.5 − 1.1) / 0.2153 ≈ 1.858. Then $P\left(\bar{X}>1.5\right)\approx 1-\Phi \left(1.858\right)\approx 0.032$P(X̄ > 1.5) ≈ 1 − Φ(1.858) ≈ 0.032. Worked out exactly from the population, the probability is 0.031, so the normal curve is close even though the population is far from normal.

The normal curve for the mean of 30 students, centered on 1.1 with standard error 0.2153. The shaded area to the right of 1.5 is about 0.032.

The usual mistakes

Using the normal curve for a single value. One student’s borrowings follow the skewed population; the theorem is about the mean of many students.

Thinking the population becomes normal. Taking samples does not change the population; only the distribution of the sample mean changes shape.

Standardizing with $\sigma$σ instead of $\frac{\sigma }{\sqrt{n}}$σ/√n. For the mean of 30 students the spread is 0.2153, not 1.18.

Trusting it for a small sample from a skewed population. The mean of 2 students above is still clearly skewed, and a normal curve would give it the wrong probabilities.