Ordinary least squares (OLS) regression is the right tool when you need to model the relationship between a continuous outcome and one or more predictor variables, and your data meets a specific set of conditions. It’s the most widely used form of linear regression, and under the right circumstances, it produces the most efficient estimates possible. But “right circumstances” is doing a lot of work in that sentence. Knowing when OLS is appropriate means understanding what it assumes about your data and recognizing when those assumptions break down.
What OLS Regression Is Designed For
OLS regression fits a straight line (or a flat plane, with multiple predictors) through your data by minimizing the sum of squared differences between the observed values and the predicted values. It works best when your outcome variable is continuous: things like income, blood pressure, temperature, or test scores. If your outcome is binary (yes/no, survived/didn’t), you’re generally better off with logistic regression. If it’s a count (number of hospital visits, number of species observed), Poisson regression is typically more appropriate.
That said, OLS can technically handle a binary outcome through what’s called a linear probability model. The predicted values estimate the probability that the outcome equals 1. This approach has known drawbacks, including predictions that fall below 0 or above 1, so logistic regression remains the standard choice for binary outcomes in most fields.
The Conditions That Make OLS Optimal
OLS isn’t just a convenient default. Under a specific set of conditions, it’s provably the best option. The Gauss-Markov theorem guarantees that when these conditions hold, OLS achieves the lowest sampling variance among all linear unbiased estimators. Statisticians call this being the “BLUE,” or best linear unbiased estimator. In practical terms, it means your estimates are as precise as they can be without introducing bias.
Those conditions are:
- The relationship is linear. The outcome variable is a linear function of the predictors (plus an error term). This doesn’t mean the predictors themselves can’t be transformed. You can include squared terms or logarithms. But the model’s parameters must combine linearly.
- Errors have a mean of zero. On average, the model doesn’t systematically over- or under-predict.
- Errors have constant variance. The spread of residuals stays the same across all levels of the predicted values. This property is called homoscedasticity.
- Errors are uncorrelated with each other. One observation’s error doesn’t predict another’s. This matters especially with time-series data or data collected from the same subjects repeatedly.
- Predictors are not perfectly correlated with each other. No predictor is an exact linear combination of the others.
When any of these break down, OLS still produces estimates, but they may no longer be the most efficient or even trustworthy. The next sections cover how to check each condition and what to do when it fails.
Checking for Linearity
The simplest diagnostic is a plot of residuals (the differences between observed and predicted values) against fitted values. If the relationship is truly linear, you’ll see residuals scattered randomly around zero with no discernible pattern. If you see a curve, a fan shape, or a U-shape, the model is missing a nonlinear relationship. A parabolic pattern, for instance, suggests you may need to add a squared term for one of your predictors. Alternatively, the relationship might call for a different modeling approach entirely.
Constant Variance of Errors
OLS assumes the errors are equally variable across all values of your predictors. If they aren’t, your coefficient estimates remain unbiased but your standard errors become unreliable, which means your confidence intervals and p-values can’t be trusted.
You can spot this visually: if a residual plot fans out (wider spread on one side than the other), variance isn’t constant. Formal tests exist as well. The Breusch-Pagan test checks whether the squared residuals are related to the predictors. Because this test is sensitive to non-normal data and small samples, a generalized version (the Koenker-Bassett test) is often used instead. The White test is another common option.
When variance clearly isn’t constant, weighted least squares (WLS) is the standard alternative. WLS gives less weight to observations with higher variance, effectively correcting for the uneven spread. For example, if each data point is an average of a different number of observations, the variance naturally differs, and you’d weight each point by how many observations went into it. If variance grows proportionally with a predictor, you’d weight inversely by that predictor’s value.
Independence of Errors
This assumption is most commonly violated in time-series data, where today’s value tends to be correlated with yesterday’s. It also comes up in repeated-measures designs, where the same person is observed multiple times. When errors are correlated, OLS underestimates the true uncertainty in your estimates.
A residual-versus-time plot is the first check: if the data are independent, residuals should look randomly scattered around zero. For a formal test, the Durbin-Watson statistic ranges from 0 to 4. A value near 2 indicates no autocorrelation. Values between 0 and 2 suggest positive autocorrelation (errors trending in the same direction), while values between 2 and 4 suggest negative autocorrelation (errors alternating direction). If you find autocorrelation, time-series models or generalized least squares are better choices than standard OLS.
Avoiding Multicollinearity
OLS requires that no predictor is a perfect linear combination of the others. In practice, perfect collinearity is rare, but high collinearity is common. When two or more predictors are strongly correlated, the model struggles to separate their individual effects. Coefficient estimates become unstable, standard errors inflate, and small changes in the data can flip the sign of a coefficient.
The variance inflation factor (VIF) quantifies this. A VIF of 1 means no correlation with other predictors. A VIF above 5 to 10 signals problematic multicollinearity. When you encounter high VIF values, your options include dropping one of the correlated predictors, combining them into a single index, or using a technique like ridge regression that’s designed to handle correlated predictors.
Dealing With Influential Outliers
Because OLS minimizes squared errors, it’s sensitive to outliers. A single extreme data point can pull the entire regression line toward it. Cook’s Distance is the most widely used measure for identifying these influential points. The standard guidelines: a Cook’s Distance greater than 0.5 warrants investigation, and a value greater than 1 strongly suggests the point is influencing your results. Even without hitting these thresholds, any point that visually stands apart from the rest of the Cook’s Distance values deserves a closer look.
If you find that your results change substantially when you remove one or two points, OLS may not be robust enough for your data. Robust regression methods that downweight extreme observations can be more appropriate in these cases.
How Much Data You Need
OLS works poorly with too few observations relative to the number of predictors. Several rules of thumb exist, and they vary by field. The most common guideline is 10 to 20 observations per predictor variable. More conservative recommendations include having at least 50 plus 8 times the number of predictors, or in ecological studies, 30 to 45 observations when studying gradients. With fewer observations than these guidelines suggest, your estimates become unstable and your model is likely to overfit, meaning it captures noise rather than real patterns.
When to Choose Something Else
OLS is the right starting point when your outcome is continuous, your sample is large enough, and the core assumptions are at least approximately met. It’s straightforward to interpret, computationally simple, and has well-understood properties. But several common situations call for a different approach:
- Binary outcome (yes/no, pass/fail): use logistic regression.
- Count outcome (number of events): use Poisson or negative binomial regression.
- Non-constant variance: use weighted least squares or robust standard errors.
- Correlated errors (time series, repeated measures): use generalized least squares, mixed models, or time-series methods.
- Highly correlated predictors: use ridge regression, lasso, or principal components regression.
- Heavy outlier influence: use robust regression methods.
The value of understanding OLS assumptions isn’t just academic. Running the diagnostics, checking the plots, and testing the conditions is what separates a regression that tells you something real from one that tells you something misleading. OLS is powerful precisely because its requirements are well defined. When you verify those requirements are met, you can trust the results. When they aren’t, you know exactly where to look for a better tool.