Outliers, Leverage & Influential Points in Regression

You’ll be able to

Distinguish a regression outlier from a high-leverage point
Predict how each affects the slope, intercept and correlation
Explain why removing a point requires justification rather than convenience

Three different kinds of unusual

A point can be unusual in two independent ways, and the exam distinguishes them. A regression outlier has a large residual — it sits far from the line vertically — but its x-value is unremarkable. A high-leverage point has an extreme x-value, far from the mean of x, regardless of whether it falls near the line. A point is influential if removing it substantially changes the slope, intercept or correlation. The clean summary: outlier is about y, leverage is about x, and influence is about consequences.

Why leverage matters so much

The least-squares line always passes through the point (x̄, ȳ), so it pivots about that point. A point far out in x acts on a long lever arm: a small vertical displacement there swings the line a great deal. A point with the same residual near the center of x barely moves it. This is why high-leverage points are the most likely to be influential, and why a scatterplot with one observation isolated far to the right deserves suspicion before any regression output is interpreted.

Effects on r and on the residual plot

An influential high-leverage point that lies along the trend can inflate the correlation dramatically, making a weak relationship look strong. One that lies off the trend can deflate r toward zero or even reverse the sign of the slope. A regression outlier with moderate x typically leaves the slope roughly intact while increasing the residual standard deviation and reducing r². The residual plot is where these show up: a lone large residual, or a single point pulling the pattern, is visible there even when the scatterplot looks acceptable.

Watch out

Never delete a point because it is inconvenient. Removing an observation requires a reason — a recording error, a subject who did not meet the study's criteria, a malfunctioning instrument. "It was hurting my r²" is not a reason, and a response that removes a point without justification loses credit even when the resulting model fits better.

Worked example

A scatterplot of ten points shows a moderate positive trend with r = 0.55. An eleventh point is added far to the right and well above the trend line. Describe its likely classification and its effect on the regression.

1.Its x-value is far from the mean of x, so it is a high-leverage point.
2.It also lies well above the line the other ten define, so it has a large residual relative to that line — it is also an outlier in the regression sense.
3.Because the line pivots about (x̄, ȳ) and this point has a long lever arm, it will pull the right end of the line upward: the slope increases and the intercept decreases.
4.Since the point extends the range of x and lies in the direction of the trend, r will most likely increase substantially, overstating the strength of the relationship.

Answer: The point is high-leverage and, relative to the original line, also a regression outlier — and it is almost certainly influential. It steepens the slope, lowers the intercept, and inflates r, so the relationship appears stronger than the other ten observations support. It should be investigated, not deleted.

Checkpoint

A point with an extreme x-value but a small residual is best described as —

Tip

The operational test for influence is direct: fit the line with the point, fit it without, and compare the slope, intercept and r. If the numbers barely move, the point is not influential however unusual it looks.

Checkpoint

The least-squares regression line always passes through which point?

Answer the 2 checkpoints as you read.