Mean and Variance of X-bar

The sample mean as a random variable

Take a sample of n values, $X_1$X₁, $X_2$X₂, up to $X_n$Xₙ, each drawn at random from a population with mean $\mu$μ and variance ${\sigma }^2$σ². Their mean is $\bar{X}=\left(X_1+X_2+\cdots +X_n\right)/n$X̄ = (X₁ + X₂ + … + Xₙ) / n, read “X-bar”.

Each $X_i$Xᵢ is a random variable with mean $\mu$μ and variance ${\sigma }^2$σ², and X̄ is a random variable too: a different sample gives a different value. Its mean and its variance can be worked out from $\mu$μ, ${\sigma }^2$σ² and n.

The mean of X̄ is $\mu$μ

Expected values add, so $E\left(X_1+X_2+\cdots +X_n\right)=\mu +\mu +\cdots +\mu =n\mu$E(X₁ + X₂ + … + Xₙ) = μ + μ + … + μ = nμ. Dividing the total by n divides its expected value by n, so $E\left(\bar{X}\right)=n\mu /n=\mu$E(X̄) = nμ / n = μ.

On average, then, the sample mean equals the population mean. A single sample mean can be too high or too low, but it does not tend to miss in either direction. An estimate with this property is called unbiased, and X̄ is an unbiased estimator of $\mu$μ.

One dice and two

Roll a fair dice. The score X is 1, 2, 3, 4, 5 or 6, each with probability $\frac{1}{6}$1/6, so $\mu =3.5$μ = 3.5. Its variance is ${\sigma }^2=\frac{35}{12}\approx 2.917$σ² = 35/12 ≈ 2.917.

Now roll two dice and take the mean of the two scores. There are 6 × 6 = 36 equally likely outcomes, and the mean can be 1, 1.5, 2, and so on up to 6. Only one outcome gives a mean of 1 (two ones), but six outcomes give 3.5. The mean of two dice is still centered on 3.5, and it is far more likely to be near 3.5 than a single score is.

Working through all 36 outcomes, the variance of the mean of two dice is $\frac{35}{24}\approx 1.458$35/24 ≈ 1.458. That is exactly half of $\frac{35}{12}$35/12: with n = 2, the variance has been divided by 2.

One dice: each score from 1 to 6 has probability $\frac{6}{36}=\frac{1}{6}$6/36 = 1/6.

The mean of two dice, in steps of 0.5 from 1 to 6, counted out of the 36 outcomes: 1, 2, 3, 4, 5, 6, 5, 4, 3, 2, 1. The center is still 3.5, and the ends are now rare.

The variance of X̄ is $\frac{{\sigma }^2}{n}$σ²/n

When the values in the sample are independent, their variances add: Var$\left(X_1+X_2+\cdots +X_n\right)={\sigma }^2+{\sigma }^2+\cdots +{\sigma }^2=n{\sigma }^2$(X₁ + X₂ + … + Xₙ) = σ² + σ² + … + σ² = nσ².

Dividing a random variable by n divides its variance by $n^2$n², because variance is measured in squared units: if every value is divided by n, every squared distance from the mean is divided by $n^2$n². So Var$\left(\bar{X}\right)=n{\sigma }^2/n^2$(X̄) = nσ² / n². One n cancels, and Var$\left(\bar{X}\right)=\frac{{\sigma }^2}{n}$(X̄) = σ²/n.

For the two dice, $\frac{{\sigma }^2}{n}=\left(\frac{35}{12}\right)/2=\frac{35}{24}$σ²/n = (35/12) / 2 = 35/24, the value found from the 36 outcomes.

One egg and twenty-five

The masses of eggs from a farm are normally distributed with mean 62 g and standard deviation 5 g, so ${\sigma }^2=25$σ² = 25. For the mean mass of 25 eggs, E(X̄) = 62 g and Var(X̄) = 25 ÷ 25 = 1, so the standard deviation of X̄ is $\sqrt{1}=1\text{g}$√1 = 1 g.

About a third of single eggs, a proportion of 0.317, weigh more than 5 g away from 62 g. The mean of 25 eggs is more than 2 g away from 62 g with probability only 0.046.

The low, wide dashed curve is the mass of one egg, with standard deviation 5 g. The tall gold curve is the mean of 25 eggs, with standard deviation 1 g. Both are centered on 62 g, and each encloses an area of 1.

Larger samples, tighter estimates

Because n is in the denominator, every extra observation makes Var(X̄) a little smaller. With ${\sigma }^2=36$σ² = 36, a sample of 4 gives Var(X̄) = 36 ÷ 4 = 9, a sample of 9 gives 36 ÷ 9 = 4, and a sample of 36 gives 36 ÷ 36 = 1.

The standard deviation of X̄ is the square root: 3, then 2, then 1. Going from 4 readings to 36, nine times as many, divides the variance by 9 but the standard deviation only by 3. The spread shrinks with the square root of n. The standard deviation of X̄, $\frac{\sigma }{\sqrt{n}}$σ/√n, is called the standard error.

When the values are not independent

The variances add only when the values in the sample are independent. That holds when sampling with replacement, and very nearly holds when the sample is a small part of a large population.

Drawing without replacement from a small population breaks it. The ten scores 3, 5, 4, 8, 2, 6, 5, 7, 4 and 6 have mean 5 and variance 3. The means of all 210 samples of four, drawn without replacement, do average 5, as $E\left(\bar{X}\right)=\mu$E(X̄) = μ says. But their variance is 0.5, not 3 ÷ 4 = 0.75: once the 8 is in a sample it cannot be drawn again, and that makes extreme samples rarer still.

The usual mistakes

Keeping the population variance. ${\sigma }^2$σ² is the variance of one value; the mean of n values has variance $\frac{{\sigma }^2}{n}$σ²/n.

Dividing the variance by $n^2$n² and stopping. The variances add to $n{\sigma }^2$nσ² first, so dividing by $n^2$n² leaves $\frac{{\sigma }^2}{n}$σ²/n.

Dividing the standard deviation by n. It is the variance that is divided by n; the standard deviation is divided by $\sqrt{n}$√n. For the eggs, $5÷\sqrt{25}=1\text{g}$5 ÷ √25 = 1 g, not $5/25=0.2\text{g}$5 / 25 = 0.2 g.