The Coefficient of Determination

The variation with no model

Five students studied for 1, 3, 5, 7 and 9 hours and scored 3, 3, 6, 5 and 8. Without using the hours at all, the best single guess for a score is the mean score, 5.

The gaps from the mean are −2, −2, 1, 0 and 3. Their squares total 4 + 4 + 1 + 0 + 9 = 18. This total, $\sum (y-$Σ(y − ȳ${)}^2$)², is the variation in y: how far the scores spread around their mean.

The level line is the mean score, 5. The gaps from the points to it, squared and added, total 18.

The variation the line leaves

The least-squares line is y = 0.6x + 2. At 1, 3, 5, 7 and 9 hours it predicts 2.6, 3.8, 5, 6.2 and 7.4, so the residuals are 0.4, −0.8, 1, −1.2 and 0.6.

Their squares total 0.16 + 0.64 + 1 + 1.44 + 0.36 = 3.6. Using the hours has cut the squared gaps from 18 to 3.6.

The same points with the regression line y = 0.6x + 2. The gaps are shorter, and their squares total 3.6.

The share explained

Of the 18, the line leaves 3.6 unexplained, a share of 3.6 ÷ 18 = 0.2. The rest, 18 − 3.6 = 14.4, is accounted for by the line.

The coefficient of determination is $R^2=1-(\text{squared residuals}÷$R² = 1 − (squared residuals ÷ squared gaps from the mean) = 1 − 3.6 ÷ 18 = 0.8. The line accounts for 80 percent of the variation in the scores, and 20 percent is left in the residuals.

$R^2$R² runs from 0 to 1. It is 0 when the model does no better than the mean, and 1 when every point lies on the model.

$R^2$R² and r

For a straight-line fit, $R^2$R² is exactly the square of Pearson’s r. Here Sxx = 40, Syy = 18 and Sxy = 24, so $r=24÷\sqrt{40\times 18}=24÷\sqrt{720}=0.894$r = 24 ÷ √(40 × 18) = 24 ÷ √720 = 0.894, and $r^2=576/720=0.8$r² = 576 / 720 = 0.8.

Square r before rounding it. Rounded first, $0.894^2=0.799$0.894² = 0.799, which is not quite 0.8.

The two numbers answer different questions. r has a sign and gives the direction and strength of the straight-line pattern. $R^2$R² has no sign, because a square cannot be negative, and gives a share of the variation. r = −0.9 and r = 0.9 both give $R^2=0.81$R² = 0.81.

Because squaring a number between 0 and 1 makes it smaller, r looks stronger than the share it explains. r = 0.7 sounds high, but $r^2=0.49$r² = 0.49: the line accounts for less than half of the variation.

A bigger model always scores higher

Least squares can fit a curve to the same five points, and an extra term can never raise the squared residuals, because the new model can always set that term to 0 and do as well as before. So $R^2$R² never falls as terms are added.

For these points the line gives $R^2=0.8$R² = 0.8, a quadratic 0.816, and a cubic 0.821. A quartic has five coefficients, enough to pass through all five points, so its squared residuals total 0 and $R^2=1$R² = 1.

The quartic is not the better model. Between and beyond the points it swings to follow every one of them: at 0 hours it predicts a score of 10.4, higher than any student scored. A larger $R^2$R² from extra terms is not evidence that the extra terms are real.

The quartic through all five points. Every residual is 0, so $R^2=1$R² = 1, but the curve dips and rises between the points and starts at 10.4 at 0 hours.

Look at the residuals too

Another five students scored 2, 5, 7, 8 and 8.5 after 1, 3, 5, 7 and 9 hours. The line y = 0.8x + 2.1 gives $R^2=0.91$R² = 0.91, which sounds like a good fit.

In order, its residuals are −0.9, 0.5, 0.9, 0.3 and −0.8: negative at both ends and positive in the middle. The points bend over, and the line cuts across the bend. A high $R^2$R² says the line follows the points closely; the pattern in the residuals says a curve would follow them better.

$R^2=0.91$R² = 0.91, but the first and last points lie below the line and the middle three above it. The gaps follow a curve.

The usual mistakes

Giving the unexplained share. 3.6 ÷ 18 = 0.2 is what the line misses; $R^2$R² is 1 − 0.2 = 0.8.

Dividing the totals the wrong way up. 18 ÷ 3.6 = 5, and a share of the variation cannot be more than 1.

Reading $R^2$R² as a share of the points. $R^2=0.64$R² = 0.64 means 64 percent of the variation in y is accounted for, not that 64 percent of the points lie on the line.

Using r as $R^2$R². For r = 0.8, $R^2=0.64$R² = 0.64, not 0.8; for r = −0.6, $R^2=0.36$R² = 0.36, not −0.36.

Flats sold in one street

In the application below, the price of ten flats is set against their floor area, with Syy = 90000 as the total variation in price. The line accounts for Sxy² ÷ Sxx of it, and the residuals hold the rest.