Modelling relationships
Regression analysis finds an equation that predicts one variable (y) from another (x). The most common is linear regression:
ŷ = a + bx (or ŷ = mx + b)
ŷ ("y-hat") is the predicted value.
Least squares regression
Many lines could pass near the data. Least squares regression picks the line that makes the sum of squared residuals as small as possible.
A residual = actual y − predicted ŷ.
- Positive residual: the point is above the line.
- Negative residual: below the line.
Squaring stops positive and negative residuals from cancelling, and penalizes big misses.
Using technology
Calculators (LinReg), spreadsheets (SLOPE, INTERCEPT, or a chart trendline) and Desmos compute the line instantly.
Example: Hours studied (x) and score (y):
| x | 1 | 2 | 3 | 4 | 5 |
|---|---|---|---|---|---|
| y | 58 | 64 | 70 | 73 | 80 |
Technology gives ŷ ≈ 53.1 + 5.3x, with r ≈ 0.995.
Interpreting the equation
- Slope (5.3): each extra hour of study is associated with about 5.3 more points, on average.
- Intercept (53.1): the predicted score with 0 hours (only meaningful if 0 is a reasonable x-value).
How good is the fit?
The coefficient of determination, R² (r² for a line), tells what fraction of the variation in y the model explains.
- R² = 0.99 → 99% explained; excellent fit.
- R² = 0.30 → only 30% explained; weak fit.
Residual plots
Plot residuals against x.
| Residual plot | Meaning |
|---|---|
| random scatter around 0 | linear model is appropriate |
| a curved pattern | a curve would fit better |
| a fan shape | spread changes with x |
Non-linear regression
When data curves, use non-linear regression:
| Model | Equation | Example |
|---|---|---|
| quadratic | y = ax² + bx + c | projectile paths |
| exponential | y = abˣ | population growth |
| power | y = axᵇ | metabolic rate vs body mass |
| logarithmic | y = a + b ln x | learning curves |
| sinusoidal | y = a sin(b(x − c)) + d | seasonal data |
Choose by looking at the scatter plot, the context, and R².
Predictions
- Interpolation (within the data’s x-range) is usually reasonable.
- Extrapolation (beyond) is risky.
Using ŷ = 53.1 + 5.3x, 3.5 h → about 71.7. But 12 h → 116.7, which is impossible on a 100-point test.
Worked examples
Example 1: ŷ = 2.4x + 10. Predict y at x = 5. 22.
Example 2: Actual y = 25 at x = 5. Residual? 25 − 22 = 3.
Example 3: A residual plot shows a U shape. What does it suggest? A non-linear (e.g., quadratic) model.
Common mistakes
- Extrapolating far beyond the data.
- Interpreting the slope as causation.
- Ignoring residual patterns.
Try it yourself
- ŷ = −3x + 50. Predict y at x = 8.
- The actual value at x = 8 is 30. Find the residual.
- R² = 0.81. What does it mean?
- Data grow by a constant percent. Which regression model?
Answers: 1) 26 2) 4 3) 81% of the variation in y is explained by the model. 4) exponential
Quick recap
- Regression analysis fits an equation to data; least squares regression minimizes squared residuals.
- Interpret the slope and check R² and residual plots.
- Use non-linear regression for curved patterns.