Comparing Two Variances with the F-Test

Two lines, two spreads

Two production lines make the same part. Their mean sizes are much the same, but line B’s parts look more variable than line A’s. The question is whether the populations differ in spread, ${\sigma }_{\text{B}}^2$σ_B² against ${\sigma }_{\text{A}}^2$σ_A², not in mean.

The chi-squared test of the last lesson compares one variance with a claimed number. Here neither variance is known: both are estimated from samples, and the two estimates are compared with each other.

A ratio, not a difference

Two spreads are compared by dividing. If both populations are normal with the same variance, the ratio of the two sample variances follows an F distribution, which depends on two numbers of degrees of freedom: one for the variance on top and one for the variance underneath.

By convention the larger sample variance goes on top: F = larger $s^2÷$s² ÷ smaller $s^2$s². Then F is at least 1, and only values far above 1 count against equal spreads.

The two samples

Line B: 10 parts, with $s_{\text{B}}^2=9.6$s_B² = 9.6. Line A: 8 parts, with $s_{\text{A}}^2=4.0$s_A² = 4.0. Each $s^2$s² divides the squared deviations from its own sample mean by one less than its sample size.

The larger is 9.6, so $F=9.6/4.0=2.4$F = 9.6 / 4.0 = 2.4. Each sample spends one degree of freedom on its own mean: line B, on top, has 10 − 1 = 9, and line A, underneath, has 8 − 1 = 7. F is read on F(9, 7), top first.

The gold curve is the F distribution with 9 and 7 degrees of freedom, highest at about 0.6. The shaded tail from 3.68 holds 5 percent of the area. The lines’ F = 2.4 is to its left.

The verdict

With 9 and 7 degrees of freedom, $P\left(F\ge 3.677\right)=0.05$P(F ≥ 3.677) = 0.05, so the tail begins at 3.68. 2.4 < 3.68, so do not reject $H_0$H₀: ${\sigma }_{\text{A}}^2={\sigma }_{\text{B}}^2$σ_A² = σ_B². The p-value is $P\left(F\ge 2.4\right)=0.1307$P(F ≥ 2.4) = 0.1307.

Stated about the lines: there is no evidence that their spreads differ. Line B’s sample variance is 2.4 times line A’s, but samples of 10 and 8 parts from equally variable lines give ratios that large about 13 times in 100.

Which level 3.68 tests at

Suppose line B was rebuilt, and the question set before measuring was whether it is now less consistent: $H_1$H₁: ${\sigma }_{\text{B}}^2>{\sigma }_{\text{A}}^2$σ_B² > σ_A². Then all 5 percent goes in the upper tail, and 3.68 is the 5 percent critical value.

Suppose instead the question was only whether the spreads differ: $H_1$H₁: ${\sigma }_{\text{A}}^2\ne {\sigma }_{\text{B}}^2$σ_A² ≠ σ_B². Either line could have come out on top. Putting the larger variance on top turns a ratio far below 1 into one far above 1, so the lower tail is never used, but each line had a 5 percent chance of landing in the upper tail. Comparing with 3.68 is then a test at 10 percent.

For that question at 5 percent, use the upper 2.5 percent point instead: $P\left(F\ge 4.823\right)=0.025$P(F ≥ 4.823) = 0.025 for 9 and 7 degrees of freedom. 2.4 is short of both 3.68 and 4.82, so the verdict here is the same at every one of these levels.

Tests for means and tests for spreads

A t-test asks whether two means differ, and an F-test asks whether two spreads differ.

The pooled two-sample t-test assumes the two populations have equal variances. An F-test that finds no evidence of a difference leaves that assumption standing; one that rejects $H_0$H₀ says the pooled test is the wrong tool for those samples.

Like the chi-squared test for one variance, the F-test assumes both populations are normal, and it is badly misled when they are not.

The usual mistakes

Putting the smaller variance on top. 4.0 ÷ 9.6 = 0.42 is a ratio below 1, and the upper-tail critical value cannot judge it.

Subtracting. 9.6 − 4.0 = 5.6 follows no F distribution; spreads are compared by their ratio.

Swapping the degrees of freedom. F(7, 9) is a different curve, with upper 5 percent point 3.293, not 3.677. The first number belongs to the variance on top.

Using standard deviations. $\sqrt{9.6}÷\sqrt{4.0}=1.55$√9.6 ÷ √4.0 = 1.55 is not F; F is a ratio of variances.

Bolts from two machines

In the application below, part (b) compares machine A, $s^2=0.08$s² = 0.08 mm² from 11 bolts, with machine B, $s^2=0.032$s² = 0.032 mm² from 13 bolts. The larger estimate goes on top, giving F = 2.5 on 10 and 12 degrees of freedom. Part (a) is the chi-squared test of the last lesson.