Scatter Plots, Correlation and Regression
☰ Contents
Here two measurements are taken from each subject instead of one, and the question becomes whether the two move together.
What does a scatter plot show?
A scatter plot shows whether two measurements on the same subjects are related. It draws one dot per subject, placed by its two measurements: the explanatory variable across and the response variable up. See Scatter Plots. Plot five students' practice hours against their scores, (1, 50), (2, 55), (3, 65), (4, 70) and (5, 80), and the dots climb from left to right. The plot helps because a relationship has a shape that a table hides: its direction, rising or falling; its form, straight or curved; and its strength, a tight band or a loose cloud. Here each extra hour adds 5 or 10 points, a positive and fairly strong linear pattern. The axis choice matters when one variable may explain the other; otherwise either way round will do. An outlier, one dot far from the rest, deserves a second look before any line is drawn. Check the rise over the whole range: 80 − 50 = 30 points over 4 extra hours.
Read all three before any calculation.
Correlation is positive when the dots rise together, negative when one measurement climbs as the other falls, and absent when the cloud has no shape. See Correlation.
Now you
What correlation do these dots show?
How do you draw a line of best fit?
Draw a line of best fit by eye through the mean point, with about as many dots above it as below. The mean point's coordinates are the mean of the x values and the mean of the y values. See Line of Best Fit. For (1, 2), (2, 3), (3, 5), (4, 6) and (5, 9), the mean of x is (1 + 2 + 3 + 4 + 5) ÷ 5 = 3 and the mean of y is (2 + 3 + 5 + 6 + 9) ÷ 5 = 5, so the line passes through (3, 5). The mean point is used because a line that balances the dots on both sides must pass through their center. Dots that curve need a different approach: no straight line fits them, so draw a curve. Read the gradient from two points on the line: through (3, 5) and (5, 8), it rises 8 − 5 = 3 over 2, a gradient of .
passing through (2, 3) forces 2m + c = 3, so each intercept c fixes m = (3 − c) / 2
Make the line pass through (2, 3)
To estimate, go up from the x value to the line and across to the y-axis, or substitute into the equation: y = 0.6x + 2.4 at x = 6 gives 6.
The residual at each point is the measured value minus the value the line predicts. Small residuals with no pattern mean the line fits. Residuals that run high, low, low, low, high mean the data curve, so a curved model fits better. See Choosing Between Two Models.
Reading the line inside the range of the data is interpolation, which the data support. Reading beyond the range is extrapolation, which nothing measured supports. See Interpolation and Extrapolation.
A line fitted to heights at ages 5 to 12 predicts a three-meter adult at age 40, and the arithmetic gives no warning.
Now you
A line of best fit is y = 2x. Estimate y when x is 6.
A line of best fit is y = 0.5x. Estimate y when x is 2.
Data that arrives in order
A time series joins readings taken in time order. To see the trend under the ups and downs, compare each low point with the one before it; to allow for the season, compare this July with last July. See Time Series.
Does correlation mean one thing causes the other?
Correlation does not show causation. Two measurements can rise together because x causes y, because y causes x, because a third variable drives both, or because a pattern in a small sample is chance. Across a year, ice-cream sales and sunburn cases rise together, but ice cream does not burn skin: hot, sunny weather drives both. A hidden driver like this is a lurking variable, and no calculation on the two measurements alone can reveal it. A strong correlation still supports prediction, even without knowing why. Evidence of cause needs a controlled experiment, in which subjects are split into groups by lot, 40 ÷ 2 = 20 in each for 40 volunteers, so that other differences are spread evenly. When an experiment is impossible, as with smoking, cause is argued from many observational studies that agree, a plausible mechanism, and larger doses giving larger effects. Test a causal claim by naming a variable that could drive both.
Shoe size and reading ability rise together in children because age drives both. See Correlation Is Not Causation.
How is correlation measured exactly?
Correlation is measured by Pearson's product-moment correlation coefficient, r, which runs from −1 to 1. Convert each measurement to standard deviations from its mean, multiply each subject's two values, and average the products. See Pearson's Correlation Coefficient. A subject above both means, or below both, gives a positive product, and a mixed subject a negative one, so the sign of r is the direction. For (1, 1), (2, 3) and (3, 2), both means are 2, the deviations are −1, 0, 1 for x and −1, 1, 0 for y, and the products total (−1) × (−1) + 0 × 1 + 1 × 0 = 1. The squared deviations total 2 for each variable, so . Because every value is scaled by its own spread, r ignores units. It judges straight lines only, so dots on a tight curve can give r near 0. Check the extreme: when y = x, r = 1.
r = 0.96 measures how tightly the points hug a line, not how steep it is: add scatter and r falls whatever the gradient
Set the scatter to 0 and flatten the line
The size of r is the strength: every dot on one rising line gives r = 1, and more scatter moves r toward 0.
A tight arch of dots can still give r near 0.
A sample r might be chance. A table gives the critical value r must beat for that sample size at the 5 percent level: ten pairs need 0.632, so r = 0.81 clears it. Falling short shows only that this sample cannot rule out chance. See Critical Values of r.
Now you
Which value of r fits these dots best?
When the relationship is not straight
Spearman's rank correlation replaces every value by its rank, its place in size order, and correlates the ranks. Any relationship that rises at every step gives 1, curved or not, and an outlier can move a rank by only a place or two. See Spearman's Rank Correlation.
Non-linear regression fits a curve by the same least-squares rule, and the shape of the data decides between quadratic, cubic, exponential, power and sine. Counts that double every two weeks need an exponential, ; the functions and logarithms guide covers that algebra. See Non-Linear Regression.
What is the regression line, and which one?
The least-squares regression line of y on x makes the sum of the squared vertical residuals as small as possible. A residual is an observed y minus the y the line predicts. Residuals below the line are negative, so they are squared before adding, or they would cancel. See The Least-Squares Regression Line. Exactly one line does this; it passes through the mean point. Take (1, 1), (2, 3) and (3, 2): Sxy is 1 and Sxx is 2, so the gradient, Sxy ÷ Sxx, is , and the line through the mean point (2, 2) is . Its residuals are , 1 and , and such residuals always total 0: . The residuals are vertical because the line predicts y from x; to predict x from y, use the line of x on y, which minimizes horizontal residuals. Try a prediction: at x = 4 the line gives 4 ÷ 2 + 1 = 3.
the quantity being minimized is Σ(y − ŷ)², an area, so a distant point counts its distance squared: 4.73
Make the total area as small as you can
Written as y = ax + b, the line has gradient a, the change in y per unit of x, and intercept b, the value of y when x is 0.
The two lines cross at the mean point and coincide only when r is 1 or −1. See The Regression Line of x on y.
A prediction inside the range of the data is a result; beyond it, a guess. The gradient is a rate for the group, not a promise about one person. See Predicting from a Regression Line.
The coefficient of determination is the share of the variation in y that the line accounts for. If the squared gaps from the mean of y total 18 and the squared residuals total 3.6, the unexplained share is and . For a straight line . A high on the wrong model only says the line follows the points it was given, so look at the residuals too. See The Coefficient of Determination.
Now you
Which total does the regression line of y on x make smallest?
Cost = 8n + 35 was fitted to n items. What is the 35?
The mistakes worth naming
- Computing r on a curved relationship. r judges straight lines only. Look at the plot first.
- Reading causation from a coefficient. No value of r rules out a lurking variable.
- Extrapolating past the data. The line has no evidence outside the range it was fitted on.
- Using the wrong regression line. Predict y with the line of y on x.
- Quoting as a percentage of the data. It is the share of the variation in y that the line accounts for.
- Ignoring sample size. An r of 0.9 from four points is weaker evidence than an r of 0.4 from four hundred.
Where this leads next
A claim about a population you did not measure is inference, in the sampling and inference guide. Means and standard deviations are in the averages and spread guide, the other displays in the charts and displays guide, and gradient and intercept in the coordinate geometry guide.
The lessons are listed in Scatter Plots and Correlation and Correlation and Regression.
Your turn
Three to try — tap what you get.
Points rising from left to right show
A correlation of −0.9 is
Correlation on its own shows
Questions and answers
- What does a correlation coefficient of 0.8 mean?
- The points lie fairly close to a straight line sloping upwards, with higher values of one variable going with higher values of the other. The coefficient runs from −1 to 1: 1 is a perfect line upwards, −1 a perfect line downwards, and 0 no linear relationship. It measures how close to linear the pattern is, not how steep the line is, and 0.8 says nothing about whether one variable causes the other.
- Why is correlation not causation?
- Because two variables can move together for at least four reasons: one causes the other, the other causes the first, a third variable causes both, or the pattern is coincidence in a small sample. Ice cream sales and drowning deaths correlate strongly, and neither causes the other; hot weather causes both. To claim causation you need a controlled experiment or a strong causal argument, not a coefficient.
- What is the least-squares regression line?
- It is the straight line that makes the total of the squared vertical distances from the points to the line as small as possible. Vertical distances are used because the line is being fitted to predict y from x, so error is measured in y. Squaring stops positive and negative errors canceling and penalises a large miss more than several small ones. The result is a unique line through the mean point of the data.
- What does r squared tell you?
- The coefficient of determination is the proportion of the variation in y that the regression on x accounts for. An r squared of 0.64 says about 64 percent of the variability in y is explained by the linear relationship with x, leaving 36 percent to everything else. It is a measure of fit, not of correctness: a high value on a badly chosen model still only says the line tracks the points you gave it.
- When should you use Spearman's rank correlation instead of Pearson's?
- Use Spearman when the relationship is monotonic but not straight, when the data are ranks or ordered judgments rather than measurements, or when outliers would distort Pearson's coefficient. Spearman replaces each value by its rank and correlates the ranks, so it detects any consistently increasing relationship, curved or not. Pearson is the better choice when the relationship really is linear and the data are genuine measurements.
- Why is extrapolation risky?
- Because a regression line is only evidence about the range of x values the data covered. Outside that range nothing has been observed, and the assumption that the same straight relationship continues is untested. Growth curves flatten, physical limits intervene, and a line fitted to children's heights between ages 5 and 12 predicts a three-meter adult. Interpolation, inside the data range, is far safer.