Scatter Plots, Correlation and Regression

Here two measurements are taken from each subject instead of one, and the question becomes whether the two move together.

What does a scatter plot show?

A scatter plot draws one dot for each subject, placed by its two measurements: the explanatory variable across and the response variable up. See Scatter Plots.

Read three things from the cloud of dots before any calculation: the direction (rising or falling), the form (straight, curved or shapeless) and the strength (a tight band or a loose drift).

Correlation is positive when the dots rise together, negative when one measurement climbs as the other falls, and absent when the cloud has no shape. See Correlation.

How do you draw a line of best fit?

The mean point has the mean of the x values as its x-coordinate and the mean of the y values as its y-coordinate. A line of best fit passes through the mean point, with about as many dots above it as below. See Line of Best Fit.

To estimate, go up from the x value to the line and across to the y-axis, or substitute into the equation: y = 0.6x + 2.4 at x = 6 gives 6.

The residual at each point is the measured value minus the value the line predicts. Small residuals with no pattern mean the line fits. Residuals that run high, low, low, low, high mean the data curve, so a curved model fits better. See Choosing Between Two Models.

Reading the line inside the range of the data is interpolation, which the data support. Reading beyond the range is extrapolation, which nothing measured supports. See Interpolation and Extrapolation.

A line fitted to heights at ages 5 to 12 predicts a three-meter adult at age 40, and the arithmetic gives no warning.

Data that arrives in order

A time series joins readings taken in time order. To see the trend under the ups and downs, compare each low point with the one before it; to allow for the season, compare this July with last July. See Time Series.

Does correlation mean one thing causes the other?

No. Two measurements can rise together because x causes y, y causes x, a third variable causes both, or the sample is small and the pattern is chance.

The third reason cannot be seen in the data. Ice-cream sales and drownings rise together because hot weather drives both; shoe size and reading ability rise together in children because age drives both. Such a hidden variable is a lurking variable. See Correlation Is Not Causation.

How is correlation measured exactly?

Pearson's product-moment correlation coefficient r converts each measurement to standard deviations from its own mean, multiplies the two converted values for each subject, and averages the products, so r does not depend on the units. See Pearson's Correlation Coefficient.

A subject above both means or below both gives a positive product and a mixed subject a negative one, so the sign of r is the direction. The size of r is the strength, from −1 to 1: every dot on one rising line gives r = 1, and more scatter moves r toward 0.

r judges straight lines only: dots on a tight curve can still give r near 0.

A sample r might be chance. A table gives the critical value r must beat for that sample size at the 5 percent level: ten pairs need 0.632, so r = 0.81 clears it. Falling short shows only that this sample cannot rule out chance. See Critical Values of r.

When the relationship is not straight

Spearman's rank correlation replaces every value by its rank, its place in size order, and correlates the ranks. Any relationship that rises at every step gives 1, curved or not, and an outlier can move a rank by only a place or two. See Spearman's Rank Correlation.

Non-linear regression fits a curve by the same least-squares rule, and the shape of the data decides between quadratic, cubic, exponential, power and sine. Counts that double every two weeks need an exponential, $y=a{b}^x$y = ab^x; the functions and logarithms guide covers that algebra. See Non-Linear Regression.

What is the regression line, and which one?

The least-squares regression line of y on x makes the sum of the squared vertical residuals as small as possible. Residuals above the line are positive and those below negative, so square them before adding, or they cancel. One line makes the total smallest, and it passes through the mean point. See The Least-Squares Regression Line.

The residuals are vertical because the line predicts y from x. Write it as y = ax + b: a is the change in y per unit of x, and b is y when x is 0.

The line of x on y makes the horizontal residuals smallest and is a different line through the same dots; the two cross at the mean point and coincide only when r is 1 or −1. Use y on x to predict y and x on y to predict x. See The Regression Line of x on y.

A prediction inside the range of the data is a result; beyond it, a guess. The gradient is a rate for the group, not a promise about one person. See Predicting from a Regression Line.

The coefficient of determination $R^2$R² is the share of the variation in y that the line accounts for. If the squared gaps from the mean of y total 18 and the squared residuals total 3.6, the unexplained share is $\frac{3.6}{18}=0.2$3.6/18 = 0.2 and $R^2=0.8$R² = 0.8. For a straight line $R^2=r^2$R² = r². A high $R^2$R² on the wrong model only says the line follows the points it was given, so look at the residuals too. See The Coefficient of Determination.

The mistakes worth naming

• Computing r on a curved relationship. r judges straight lines only. Look at the plot first.
• Reading causation from a coefficient. No value of r rules out a lurking variable.
• Extrapolating past the data. The line has no evidence outside the range it was fitted on.
• Using the wrong regression line. Predict y with the line of y on x.
• Quoting $R^2$R² as a percentage of the data. It is the share of the variation in y that the line accounts for.
• Ignoring sample size. An r of 0.9 from four points is weaker evidence than an r of 0.4 from four hundred.

Where this leads next

A claim about a population you did not measure is inference, in the sampling and inference guide. Means and standard deviations are in the averages and spread guide, the other displays in the charts and displays guide, and gradient and intercept in the coordinate geometry guide.

Learn this properly in the app

Math Challenge teaches each lesson above with a drawn scatter plot, a worked calculation and practice questions, in Scatter Plots and Correlation and Correlation and Regression.