Two variables
When each individual has two measurements, a scatterplot shows the relationship. We call the predictor the explanatory variable (x) and the outcome the response variable (y).
Correlation
The correlation coefficient r ranges from −1 to +1.
- r near +1: strong positive linear association.
- r near −1: strong negative linear association.
- r near 0: little or no linear association. A curved pattern can have r near 0 yet a clear relationship.
r has no units and does not change if you swap x and y. It is sensitive to outliers. And once again: correlation does not prove causation.
The least-squares regression line
The line ŷ = a + bx is chosen to minimise the sum of the squared residuals, where a residual is actual y minus predicted ŷ.
- Slope: b = r · (s_y / s_x). It is the predicted change in y for each one-unit increase in x.
- Intercept: a = ȳ − b · x̄. It is the predicted y when x = 0 (it may not make sense in context).
- The line always passes through the point (x̄, ȳ).
Coefficient of determination
r² is the proportion of the variation in y explained by the linear relationship with x. With r = 0.8, r² = 0.64, so 64 percent of the variation in y is accounted for.
Prediction and extrapolation
Plug x into the line to predict y. This is reliable only within the range of the observed x-values. Extrapolation, predicting far outside that range, is risky because the pattern may not continue.
Worked example 1: building the line
For study data, x = hours studied and y = test score. Summary statistics: x̄ = 20, ȳ = 50, s_x = 5, s_y = 10, r = 0.6.
- Slope: b = 0.6 × (10 / 5) = 1.2 points per extra hour.
- Intercept: a = 50 − 1.2 × 20 = 50 − 24 = 26.
- Line: ŷ = 26 + 1.2x.
- Prediction at x = 25: ŷ = 26 + 30 = 56.
Worked example 2: residuals
Suppose another fitted line is ŷ = 40 + 5x for hours studied and score.
- At x = 6: ŷ = 40 + 30 = 70.
- If the student actually scored 75, the residual = 75 − 70 = +5; the line under-predicted.
- Interpretation of slope: each additional hour of study is associated with about 5 more points, on average.
Checking the fit
Plot the residuals. They should scatter randomly around zero. A curved pattern means a straight line is the wrong model. Watch for influential points, which can pull the line toward them.
Common mistakes
- Reading a high r as proof that x causes y.
- Extrapolating beyond the data.
- Interpreting the slope as the change in x per unit of y.