The Two-Sample T-Test

Two groups of plants

Sixteen seedlings of the same kind are split at random into two groups of 8. One group gets plain water, the other water with plant food. After six weeks their heights, in centimeters, are:

plain: 16, 17, 18, 20, 20, 21, 23, 25, with mean 160 ÷ 8 = 20 cm;

fed: 19, 22, 22, 23, 25, 26, 27, 28, with mean 192 ÷ 8 = 24 cm.

The fed plants are 4 cm taller on average. But the two groups overlap a lot, and two groups of plain-watered plants would also have different means. The question is whether 4 cm is more than that kind of chance would give.

The eight plants given plain water, one dot each. Their heights run from 16 cm to 25 cm, with mean 20 cm.

The eight plants given plant food, on the same scale. Their heights run from 19 cm to 28 cm, with mean 24 cm.

The hypotheses

Let ${\mu }_1$μ₁ be the mean height of all such seedlings grown with plain water, and ${\mu }_2$μ₂ the mean with plant food. $H_0$H₀: ${\mu }_1={\mu }_2$μ₁ = μ₂. Under $H_0$H₀, both groups come from populations with one shared mean, and the 4 cm gap is sampling variation.

The question asked before the trial was whether the food makes the plants taller, so $H_1$H₁: ${\mu }_2>{\mu }_1$μ₂ > μ₁, a one-tailed test at 5 percent. Had the question been only whether the food makes a difference, $H_1$H₁ would be ${\mu }_1\ne {\mu }_2$μ₁ ≠ μ₂, a two-tailed test.

Spread from the samples

A z-test would divide the gap by a standard error built from the population standard deviation. Nobody knows it for these seedlings, so the spread has to be estimated from the two samples, as s was in the one-sample t-test.

For the plain group, the deviations from 20 are −4, −3, −2, 0, 0, 1, 3 and 5. Their squares add up to 16 + 9 + 4 + 0 + 0 + 1 + 9 + 25 = 64, so $s_1^2=\frac{64}{7}=9.143$s₁² = 64/7 = 9.143.

For the fed group, the deviations from 24 are −5, −2, −2, −1, 1, 2, 3 and 4. Their squares add up to 25 + 4 + 4 + 1 + 1 + 4 + 9 + 16 = 64, so $s_2^2=\frac{64}{7}=9.143$s₂² = 64/7 = 9.143 as well.

One pooled variance

The pooled test assumes the two populations have the same variance, and combines the two estimates into one, each weighted by its degrees of freedom: pooled $s^2=\frac{\left(n_1-1\right)s_1^2+\left(n_2-1\right)s_2^2}{n_1+n_2-2}=\frac{64+64}{14}=9.143$s² = ((n₁ − 1)s₁² + (n₂ − 1)s₂²)/(n₁ + n₂ − 2) = (64 + 64)/14 = 9.143.

Each sample spends one degree of freedom on its own mean, so the pooled estimate has $n_1+n_2-2=8+8-2=14$n₁ + n₂ − 2 = 8 + 8 − 2 = 14 degrees of freedom, and so does the test.

The gap in standard errors

The standard error of the gap between the two sample means is $\surd ($√(pooled $s^2\times \left(\frac{1}{n_1}+\frac{1}{n_2}\right))=\sqrt{9.143\times \left(\frac{1}{8}+\frac{1}{8}\right)}=\sqrt{2.286}=1.512$s² × (1/n₁ + 1/n₂)) = √(9.143 × (1/8 + 1/8)) = √2.286 = 1.512 cm.

So $t=\frac{{\bar{x}}_2-{\bar{x}}_1}{1.512}=\frac{24-20}{1.512}=2.646$t = (x̄₂ − x̄₁)/1.512 = (24 − 20)/1.512 = 2.646. The fed mean is 2.646 standard errors above the plain mean.

The t distribution with 14 degrees of freedom. The shaded tail beyond the plants’ t = 2.646 is the p-value, 0.0096: under one percent of the area.

From t to p

The p-value is the chance, if $H_0$H₀ is true, of a t at least this large: $P\left(T\ge 2.646\right)=0.0096$P(T ≥ 2.646) = 0.0096 on 14 degrees of freedom, about 0.01. A calculator gives it directly from the two lists.

0.0096 < 0.05, so reject $H_0$H₀. Equally, the one-tailed 5 percent critical value is 1.761, and 2.646 is beyond it.

Stated about the plants: there is evidence at the 5 percent level that seedlings given plant food grow taller, on average, than seedlings given plain water.

With the two-tailed question, ${\mu }_1\ne {\mu }_2$μ₁ ≠ μ₂, the p-value doubles to 0.0192 and the critical value is 2.145. $H_0$H₀ is rejected at 5 percent either way.

What the test assumes

The samples must be independent: each plant is in one group only, and one plant’s height says nothing about another’s. The same plants measured before and after feeding would not be two independent samples.

Each population should be roughly normal. With only eight plants a group, that is assumed, not checked.

The pooled form also assumes equal population variances. Here both sample variances are 9.143, so nothing argues against it. When the two sample variances are very different, the F-test, two lessons on, tests whether the populations’ spreads differ.

The sample sizes do not need to be equal. With $n_1$n₁ and $n_2$n₂ different, the same formulas hold, and the degrees of freedom are still $n_1+n_2-2$n₁ + n₂ − 2.

The usual mistakes

Using one sample’s degrees of freedom. 8 − 1 = 7 belongs to a one-sample test; the pooled estimate uses both samples, which gives 14.

Adding the two variances without weighting or dividing. 9.143 + 9.143 = 18.29 is not the pooled variance; the pooled variance is their weighted average.

Dividing the gap by the pooled standard deviation instead of by the standard error of the gap. $\sqrt{9.143}=3.02$√9.143 = 3.02 cm is the spread of single plants; the gap between two means of 8 varies less.

Saying the means are proved equal when $H_0$H₀ is not rejected. The test only finds whether the data give enough evidence against $H_0$H₀.

Tomato seedlings and two fertilizers

In the application below, 6 seedlings get fertilizer A and 6 get fertilizer B. The two variance estimates are pooled, the test is two-tailed on 6 + 6 − 2 = 10 degrees of freedom, and the verdict is taken at 5 percent and at 1 percent.