Regression Analysis

What this unit covers

The topics below follow the published Statistics course framework for Unit 5. This unit is worth 10–20% of the exam, so budget your time against that rather than against how long the unit takes to teach.

Lessons in this unit

• Sampling Distributions & Variability13 min · 3 objectivesDefine a sampling distribution as the distribution of a statistic over all samples · Explain what it means for a statistic to be an unbiased estimator · Describe how sample size affects the variability of a statistic
• The Sampling Distribution of a Sample Mean15 min · 3 objectivesState the mean and standard deviation of the sampling distribution of x-bar · Apply the Central Limit Theorem to determine the shape of the distribution of x-bar · Compute probabilities involving a sample mean
• The Sampling Distribution of a Sample Proportion14 min · 3 objectivesState the mean and standard deviation of the sampling distribution of p-hat · Check the conditions for approximate Normality of p-hat · Compute probabilities involving a sample proportion
• Differences & the Big Picture of Variability13 min · 3 objectivesFind the mean and standard deviation of a difference between two independent statistics · Explain how variability of a difference combines the two individual variabilities · Connect sampling distributions to the logic of inference
• Three Distributions That Are Easy to Confuse14 min · 3 objectivesDistinguish the population distribution, the distribution of one sample, and the sampling distribution · Identify which of the three a given statement or graph refers to · Explain what each distribution is used for in inference
• Why n ≥ 30 Is a Rule of Thumb, Not a Theorem14 min · 3 objectivesState the Central Limit Theorem precisely and identify what it does and does not claim · Explain why the required sample size depends on the population shape · Check the conditions for Normality of a sampling distribution

Formulas in Unit 5

Every term in Unit 5

All 45 terms we publish for Regression Analysis, with definitions. Reading them through is the fastest way to find the ones you cannot define — then drill those in cram mode until you can produce them without the prompt.

Residual

Actual y minus predicted ŷ. Positive means the point lies above the line; a curved residual plot signals a poor linear fit.

Explanatory vs response variable

The explanatory variable goes on the x-axis and is thought to explain changes in the response on the y-axis.

Describing a scatterplot

Direction, form, strength and unusual features, in context. All four are required for full credit.

Correlation coefficient r

Measures the strength and direction of a LINEAR relationship, between −1 and 1. It has no units and is unaffected by which variable is x.

What r does not tell you

It does not establish causation, does not detect curved relationships, and is not resistant to outliers.

Least-squares regression line

ŷ = a + bx, minimizing the sum of squared residuals. It always passes through the point (x̄, ȳ).

Interpreting the slope

"For each additional one-unit increase in x, the predicted y increases by b units" — predicted, in context, with units.

Interpreting the y-intercept

The predicted response when x = 0. Often meaningless in context, and worth saying so.

Coefficient of determination r²

The percentage of variation in y explained by the linear relationship with x. Always report it as a percentage of variation.

Residual plot

A random scatter about zero supports a linear model; a curved pattern means the relationship is not linear and a different model is needed.

Influential point vs outlier

An outlier lies far from the pattern; an influential point substantially changes the regression line, usually because its x-value is extreme.

Extrapolation

Predicting outside the range of the observed x-values. Unreliable, because there is no evidence the pattern continues.

Transforming to achieve linearity

Taking logs of y linearises exponential relationships; logs of both linearises power relationships.

Population regression model

y = α + βx + ε, with ε normally distributed with constant standard deviation. Inference is about β, the true slope.

Conditions for slope inference (LINER)

Linear relationship, Independent observations, Normal residuals, Equal variance about the line, and Random data collection.

Checking conditions with a residual plot

No curved pattern supports linearity; constant vertical spread supports equal variance. A histogram of residuals checks normality.

t-interval for the slope

b ± t*·SE(b) with df = n − 2, interpreted as capturing the true slope in context and units.

t-test for the slope

H₀: β = 0 means no linear relationship. t = b/SE(b) with n − 2 degrees of freedom.

Reading regression output

The coefficient column gives a and b, the SE column gives SE(b), and the printed t and p-value are for the two-sided test of zero slope.

Why df = n − 2

Two parameters, the slope and the intercept, are estimated from the data, so two degrees of freedom are used up.

Interpreting a significant slope

Convincing evidence of a linear relationship in the population — not evidence of causation unless the data came from a randomized experiment.

Interpreting r in context

State the direction, the strength and that the relationship is linear, naming both variables. r alone is not an interpretation.

Predicting with a regression equation

Substitute and label the answer as a predicted value. Saying it will happen overstates what a model provides.

Residual for a specific point

Observed minus predicted at that x. A positive residual means the model underpredicted that observation.

Why r² is not r squared conceptually

r² is the proportion of variation in the response explained by the model, which is a different statement from the strength of the linear association.

Effect of removing an influential point

Recompute and compare. An influential point that pulls the line toward itself will change the slope noticeably when removed.

Choosing a transformation

A curved residual plot means the linear model is wrong. Try log(y) for exponential growth and log-log for a power relationship.

Standard error of the slope

Measures how much the estimated slope would vary across repeated samples. Read it from the SE Coef column of regression output.

Why the printed p-value may need halving

Software reports a two-sided p-value. A one-sided alternative uses half of it, provided the sample slope is in the hypothesized direction.

Confidence interval for the slope in context

State that you are confident the interval captures the true average change in the response per one-unit increase in the explanatory variable, with units.

Extrapolation warning in slope inference

Inference applies only over the range of x values observed; predictions beyond it have no support from the data.

Direction, form, strength

The three things a scatterplot description must contain, plus unusual features and context. Direction is positive or negative, form is linear or curved, strength is how tightly the points follow the pattern.

r is unitless and bounded

Correlation lies between −1 and 1 and has no units, because it is computed from standardized values. Changing the units of either variable leaves it unchanged.

Swapping x and y

The correlation is unchanged, because it is symmetric in the two variables. The regression line is NOT — predicting y from x gives a different line from predicting x from y.

Lurking variable

A variable not included in the study that influences both the explanatory and response variables, creating an association that is not causal. Naming a plausible one is what an "explain why causation cannot be concluded" question wants.

Slope from correlation

b = r(s_y/s_x). The slope carries the units of y per unit of x while r does not, which is why the ratio of standard deviations appears.

Sum of the residuals

Always zero for a least-squares line, by construction. This is why the mean residual carries no information and the residual PLOT, not the residual total, is what diagnoses a poor fit.

Sign of a residual

Positive means the actual value exceeded the prediction — the model underestimated. Negative means the model overestimated. Stating which way the model erred is worth a point.

High-leverage point

A point with an extreme x-value, far from x̄. Because the line pivots about (x̄, ȳ), such a point has a long lever arm and is the most likely kind to be influential.

Testing whether a point is influential

Fit the line with and without it and compare the slope, intercept and r. If they barely move it is not influential, however unusual it looks on the scatterplot.

Sampling distribution of the slope

Over repeated samples, the least-squares slope b varies around the true slope β. Inference for slopes is inference about that sampling distribution, exactly as for a mean.

Hypotheses for slope inference

H₀: β = 0 against a one- or two-sided alternative. A slope of zero means no linear relationship, so the test asks whether the observed association could be chance.

Reading standard error from output

Regression output lists the coefficient and its standard error in adjacent columns; t is their ratio. Confirming that t equals b divided by SE(b) is a quick check you read the right row.

Why slope inference uses df = n − 2

Two parameters, the slope and the intercept, are estimated from the data before the residuals are computed, so two degrees of freedom are spent.

Residual plot patterns

Random scatter supports linearity and constant variance. Curvature indicates the wrong model; a funnel shape indicates non-constant variance, which invalidates the standard errors.

What examiners penalize here

• Before finding a probability for x-bar, justify the Normal shape: either state the *population* is Normal, or invoke the **CLT with n ≥ 30**. Free-response answers that skip this justification lose the condition point even if the arithmetic is perfect.
• Every two-sample standard error you will meet is built by the same move: compute each group’s variance contribution, **add** them, then square-root. Memorize the pattern once and it powers two-proportion and two-mean intervals and tests alike.
• A question that says "the distribution of sample means" or "if we repeated this many times" is about the **sampling** distribution. A question that says "the heights of the 40 students in the sample" is about the **sample**. Reading which one is being asked about is often the whole item.

Practice Statistics

Our practice bank is drawn from across the whole course rather than filtered to one unit, which is closer to how the exam asks anyway — it will not tell you which unit a question is testing.

Questions about this unit

All 5 units of AP Statistics

1. Unit 1 · Exploring One-Variable Data and Collecting Data
2. Unit 2 · Probability, Random Variables, and Probability Distributions
3. Unit 3 · Inference for Categorical Data: Proportions
4. Unit 4 · Inference for Quantitative Data: Means
5. Unit 5 · Regression Analysis