Scatter Plots, Correlation and Regression

☰ Contents

Here two measurements are taken from each subject instead of one, and the question becomes whether the two move together.

What does a scatter plot show?

A scatter plot shows whether two measurements on the same subjects are related. It draws one dot per subject, placed by its two measurements: the explanatory variable across and the response variable up. See Scatter Plots. Plot five students' practice hours against their scores, (1, 50), (2, 55), (3, 65), (4, 70) and (5, 80), and the dots climb from left to right. The plot helps because a relationship has a shape that a table hides: its direction, rising or falling; its form, straight or curved; and its strength, a tight band or a loose cloud. Here each extra hour adds 5 or 10 points, a positive and fairly strong linear pattern. The axis choice matters when one variable may explain the other; otherwise either way round will do. An outlier, one dot far from the rest, deserves a second look before any line is drawn. Check the rise over the whole range: 80 − 50 = 30 points over 4 extra hours.

hours practicedscore
Each dot is one person: hours practiced on the x-axis, score on the y-axis. Full lesson: Scatter Plots

Read all three before any calculation.

Correlation is positive when the dots rise together, negative when one measurement climbs as the other falls, and absent when the cloud has no shape. See Correlation.

hours of TVscore
As hours of TV go up, the score goes down. That is negative correlation. Full lesson: Correlation
shoe sizescore
A shapeless cloud is no correlation: one measurement tells you nothing about the other. Full lesson: Correlation
Look at the plot before computing anything. Frank Anscombe's four data sets share identical means, standard deviations, correlation and regression line, yet one is a straight band, one a smooth curve, one a perfect line with an outlier, and one a vertical stack with a single far point.

Now you

shoe sizescore

What correlation do these dots show?

How do you draw a line of best fit?

Draw a line of best fit by eye through the mean point, with about as many dots above it as below. The mean point's coordinates are the mean of the x values and the mean of the y values. See Line of Best Fit. For (1, 2), (2, 3), (3, 5), (4, 6) and (5, 9), the mean of x is (1 + 2 + 3 + 4 + 5) ÷ 5 = 3 and the mean of y is (2 + 3 + 5 + 6 + 9) ÷ 5 = 5, so the line passes through (3, 5). The mean point is used because a line that balances the dots on both sides must pass through their center. Dots that curve need a different approach: no straight line fits them, so draw a curve. Read the gradient from two points on the line: through (3, 5) and (5, 8), it rises 8 − 5 = 3 over 2, a gradient of 3/2.

12345−224xy

passing through (2, 3) forces 2m + c = 3, so each intercept c fixes m = (3 − c) / 2

Make the line pass through (2, 3)

xymean
Draw the line through the mean point, with roughly as many dots above it as below. Full lesson: Line of Best Fit

To estimate, go up from the x value to the line and across to the y-axis, or substitute into the equation: y = 0.6x + 2.4 at x = 6 gives 6.

The residual at each point is the measured value minus the value the line predicts. Small residuals with no pattern mean the line fits. Residuals that run high, low, low, low, high mean the data curve, so a curved model fits better. See Choosing Between Two Models.

weekplants
The gap at each point is the residual: the count minus what the line predicted. Full lesson: Choosing Between Two Models

Reading the line inside the range of the data is interpolation, which the data support. Reading beyond the range is extrapolation, which nothing measured supports. See Interpolation and Extrapolation.

xy
Extending the line to 20 is extrapolation: no data there supports it. Full lesson: Interpolation and Extrapolation

A line fitted to heights at ages 5 to 12 predicts a three-meter adult at age 40, and the arithmetic gives no warning.

Now you

xy

A line of best fit is y = 2x. Estimate y when x is 6.

xy

A line of best fit is y = 0.5x. Estimate y when x is 2.

Data that arrives in order

A time series joins readings taken in time order. To see the trend under the ups and downs, compare each low point with the one before it; to allow for the season, compare this July with last July. See Time Series.

q1q2q3q4q5q6q7q8
It goes up and down, but each low point is higher than the one before: the overall trend is rising. Full lesson: Time Series

Does correlation mean one thing causes the other?

Correlation does not show causation. Two measurements can rise together because x causes y, because y causes x, because a third variable drives both, or because a pattern in a small sample is chance. Across a year, ice-cream sales and sunburn cases rise together, but ice cream does not burn skin: hot, sunny weather drives both. A hidden driver like this is a lurking variable, and no calculation on the two measurements alone can reveal it. A strong correlation still supports prediction, even without knowing why. Evidence of cause needs a controlled experiment, in which subjects are split into groups by lot, 40 ÷ 2 = 20 in each for 40 volunteers, so that other differences are spread evenly. When an experiment is impossible, as with smoking, cause is argued from many observational studies that agree, a plausible mechanism, and larger doses giving larger effects. Test a causal claim by naming a variable that could drive both.

Shoe size and reading ability rise together in children because age drives both. See Correlation Is Not Causation.

ice-cream salesdrownings
Ice-cream sales and drownings rise together: the correlation is real. Full lesson: Correlation Is Not Causation
hot weatherice cream sellspeople swim
Behind both sits a shared cause: hot weather sells ice cream and sends people swimming. Full lesson: Correlation Is Not Causation
Only an experiment rules out a lurking variable, because the experimenter sets the explanatory variable and assigns treatments at random. A study that only observes reports an association, not a cause.

How is correlation measured exactly?

Correlation is measured by Pearson's product-moment correlation coefficient, r, which runs from −1 to 1. Convert each measurement to standard deviations from its mean, multiply each subject's two values, and average the products. See Pearson's Correlation Coefficient. A subject above both means, or below both, gives a positive product, and a mixed subject a negative one, so the sign of r is the direction. For (1, 1), (2, 3) and (3, 2), both means are 2, the deviations are −1, 0, 1 for x and −1, 1, 0 for y, and the products total (−1) × (−1) + 0 × 1 + 1 × 0 = 1. The squared deviations total 2 for each variable, so r = 1/√(2 × 2) = 1/2. Because every value is scaled by its own spread, r ignores units. It judges straight lines only, so dots on a tight curve can give r near 0. Check the extreme: when y = x, r = 1.

r = 0.96gradient 1gradientscatter

r = 0.96 measures how tightly the points hug a line, not how steep it is: add scatter and r falls whatever the gradient

Set the scatter to 0 and flatten the line

The size of r is the strength: every dot on one rising line gives r = 1, and more scatter moves r toward 0.

hours studiedscore
The same rise with more scatter drops r toward 0. The size of r is the strength. Full lesson: Pearson’s Correlation Coefficient

A tight arch of dots can still give r near 0.

water givencrop
And r judges straight lines only — this tight arch still gives r near 0. Full lesson: Pearson’s Correlation Coefficient

A sample r might be chance. A table gives the critical value r must beat for that sample size at the 5 percent level: ten pairs need 0.632, so r = 0.81 clears it. Falling short shows only that this sample cannot rule out chance. See Critical Values of r.

00.250.50.751barr
Ten pairs put the bar at 0.632, and an r of 0.81 clears it comfortably. Full lesson: Critical Values of r

Now you

shoe sizescore

Which value of r fits these dots best?

When the relationship is not straight

Spearman's rank correlation replaces every value by its rank, its place in size order, and correlates the ranks. Any relationship that rises at every step gives 1, curved or not, and an outlier can move a rank by only a place or two. See Spearman's Rank Correlation.

rank xrank y
Ranking turns the curved pattern into a straight line, and two orders that agree perfectly give a rank correlation of 1. Full lesson: Spearman’s Rank Correlation

Non-linear regression fits a curve by the same least-squares rule, and the shape of the data decides between quadratic, cubic, exponential, power and sine. Counts that double every two weeks need an exponential, y = ab^x; the functions and logarithms guide covers that algebra. See Non-Linear Regression.

water givencrop
Fit a quadratic by the same rule and every gap closes: a straight line was the wrong shape for these data. Full lesson: Non-Linear Regression

What is the regression line, and which one?

The least-squares regression line of y on x makes the sum of the squared vertical residuals as small as possible. A residual is an observed y minus the y the line predicts. Residuals below the line are negative, so they are squared before adding, or they would cancel. See The Least-Squares Regression Line. Exactly one line does this; it passes through the mean point. Take (1, 1), (2, 3) and (3, 2): Sxy is 1 and Sxx is 2, so the gradient, Sxy ÷ Sxx, is 1/2, and the line through the mean point (2, 2) is y = x/2 + 1. Its residuals are −1/2, 1 and −1/2, and such residuals always total 0: −1/2 + 1 − 1/2 = 0. The residuals are vertical because the line predicts y from x; to predict x from y, use the line of x on y, which minimizes horizontal residuals. Try a prediction: at x = 4 the line gives 4 ÷ 2 + 1 = 3.

total area 4.73

the quantity being minimized is Σ(y − ŷ)², an area, so a distant point counts its distance squared: 4.73

Make the total area as small as you can

hours studiedscore
Tilt the line and some gaps shrink while others grow. Gaps above and below the line have opposite signs, so square them before adding, or they cancel. Full lesson: The Least-Squares Regression Line

Written as y = ax + b, the line has gradient a, the change in y per unit of x, and intercept b, the value of y when x is 0.

The two lines cross at the mean point and coincide only when r is 1 or −1. See The Regression Line of x on y.

hours studiedscore
Back on the original axes, the fit of x on y is a different line. Full lesson: The Regression Line of x on y

A prediction inside the range of the data is a result; beyond it, a guess. The gradient is a rate for the group, not a promise about one person. See Predicting from a Regression Line.

The coefficient of determination R² is the share of the variation in y that the line accounts for. If the squared gaps from the mean of y total 18 and the squared residuals total 3.6, the unexplained share is 3.6/18 = 0.2 and R² = 0.8. For a straight line R² = r². A high R² on the wrong model only says the line follows the points it was given, so look at the residuals too. See The Coefficient of Determination.

hours studiedscore
R² here is 0.91, but the residuals still curve, so look at them as well as at R². Full lesson: The Coefficient of Determination

Now you

hours studiedscore

Which total does the regression line of y on x make smallest?

b = 35b is the interceptthe cost at n = 0

Cost = 8n + 35 was fitted to n items. What is the 35?

The mistakes worth naming

Where this leads next

A claim about a population you did not measure is inference, in the sampling and inference guide. Means and standard deviations are in the averages and spread guide, the other displays in the charts and displays guide, and gradient and intercept in the coordinate geometry guide.

The lessons are listed in Scatter Plots and Correlation and Correlation and Regression.

Your turn

Three to try — tap what you get.

Points rising from left to right show

A correlation of −0.9 is

Correlation on its own shows

Questions and answers

What does a correlation coefficient of 0.8 mean?
The points lie fairly close to a straight line sloping upwards, with higher values of one variable going with higher values of the other. The coefficient runs from −1 to 1: 1 is a perfect line upwards, −1 a perfect line downwards, and 0 no linear relationship. It measures how close to linear the pattern is, not how steep the line is, and 0.8 says nothing about whether one variable causes the other.
Why is correlation not causation?
Because two variables can move together for at least four reasons: one causes the other, the other causes the first, a third variable causes both, or the pattern is coincidence in a small sample. Ice cream sales and drowning deaths correlate strongly, and neither causes the other; hot weather causes both. To claim causation you need a controlled experiment or a strong causal argument, not a coefficient.
What is the least-squares regression line?
It is the straight line that makes the total of the squared vertical distances from the points to the line as small as possible. Vertical distances are used because the line is being fitted to predict y from x, so error is measured in y. Squaring stops positive and negative errors canceling and penalises a large miss more than several small ones. The result is a unique line through the mean point of the data.
What does r squared tell you?
The coefficient of determination is the proportion of the variation in y that the regression on x accounts for. An r squared of 0.64 says about 64 percent of the variability in y is explained by the linear relationship with x, leaving 36 percent to everything else. It is a measure of fit, not of correctness: a high value on a badly chosen model still only says the line tracks the points you gave it.
When should you use Spearman's rank correlation instead of Pearson's?
Use Spearman when the relationship is monotonic but not straight, when the data are ranks or ordered judgments rather than measurements, or when outliers would distort Pearson's coefficient. Spearman replaces each value by its rank and correlates the ranks, so it detects any consistently increasing relationship, curved or not. Pearson is the better choice when the relationship really is linear and the data are genuine measurements.
Why is extrapolation risky?
Because a regression line is only evidence about the range of x values the data covered. Outside that range nothing has been observed, and the assumption that the same straight relationship continues is untested. Growth curves flatten, physical limits intervene, and a line fitted to children's heights between ages 5 and 12 predicts a three-meter adult. Interpolation, inside the data range, is far safer.
Mr. Chalk