Linear regression is used whenever you need to understand or predict how one measurable thing changes in response to another. It’s one of the most widely applied statistical methods across science, business, and medicine because it answers a straightforward question: if X goes up, what happens to Y, and by how much? The technique works by fitting a straight line through data points to capture that relationship, making it useful for both explanation and prediction.
The Core Idea Behind Linear Regression
At its simplest, linear regression takes a set of data points and draws the best-fitting straight line through them. That line lets you estimate the value of something you care about (the outcome) based on something you can measure (the predictor). If you’re looking at how advertising spending relates to sales revenue, for instance, the regression line tells you roughly how much additional revenue you’d expect for each additional dollar spent on ads.
Simple linear regression uses one predictor variable. Multiple linear regression uses two or more. The decision to move from simple to multiple regression depends on the question you’re asking. If you want to isolate how a single factor relates to an outcome, simple regression works. But most real-world outcomes are shaped by several factors simultaneously. Predicting a patient’s recovery time, for example, requires accounting for age, pre-existing conditions, and medication all at once. Multiple regression handles that by estimating each factor’s individual contribution while holding the others constant.
Where Linear Regression Shows Up in Practice
Linear regression is a workhorse across industries because the underlying logic, predicting an outcome from measurable inputs, applies almost everywhere.
- Real estate: Platforms predict home prices using square footage, number of bedrooms and bathrooms, location, property age, and local market conditions as inputs.
- Finance: Banks predict loan default risk by analyzing income, credit history, debt-to-income ratio, and employment stability. Investment analysts use it to measure how sensitive a stock is to overall market movements, a concept called beta in the Capital Asset Pricing Model.
- Healthcare: Hospitals predict patient recovery times or treatment effectiveness based on age, pre-existing conditions, and dosages. Epidemiologists use it to detect dose-response relationships, testing whether higher exposure to a substance corresponds to worse health outcomes.
- Marketing and retail: Companies forecast future sales by feeding in historical sales data, advertising expenses, promotions, seasonality, and economic indicators.
- Manufacturing: Factories forecast product demand using historical sales patterns, seasonal trends, and economic conditions to plan production schedules.
In each case, the goal is the same: quantify the relationship between inputs and an outcome so you can either explain what’s driving results or predict future ones.
When Linear Regression Is the Right Choice
Linear regression is appropriate when several conditions hold. The relationship between your variables needs to be roughly linear, meaning a straight line is a reasonable description of how they relate. Your data points should be independent of each other, so one observation doesn’t influence another. The spread of your data around the line should stay relatively consistent across the range (statisticians call this homoscedasticity, but it just means the errors don’t fan out or squeeze together as you move along the line). And for formal statistical testing, the errors should follow a bell-curve distribution.
When these conditions are met, the line you get from linear regression has a special property: it’s the most precise unbiased estimate possible among all methods that combine your data in a linear way. That mathematical guarantee, known as the Gauss-Markov theorem, is part of why the technique remains so popular even as flashier methods emerge.
How to Tell If Your Model Is Working
The most common measure of fit is R-squared, which tells you what percentage of the variation in your outcome is explained by your predictors. An R-squared of 0.80 means 80% of the variation is accounted for. But what counts as “good” depends entirely on the field.
In the physical sciences and engineering, R-squared values above 0.70 are generally expected, and chemistry or physics models often aim for 0.70 to 0.99. Finance models typically fall in the 0.40 to 0.70 range. In social sciences and psychology, where human behavior introduces enormous variability, values as low as 0.10 to 0.30 are often considered acceptable. Comparing your R-squared to standards from a completely different field will mislead you.
Beyond R-squared, residual plots are the most useful diagnostic tool. Residuals are the gaps between what your model predicted and what actually happened. When you plot these gaps against the predicted values, you want to see a random scatter with no pattern. If you see a curve, that’s evidence the relationship isn’t linear and a straight line is missing something important. If the scatter widens or narrows across the plot, the spread of your errors isn’t constant, which undermines the reliability of your predictions. Both problems can show up simultaneously.
When Linear Regression Fails
Linear regression produces misleading results when the true relationship between variables isn’t a straight line. Three common patterns cause trouble. Exponential growth, like bacterial populations doubling at regular intervals, curves upward in a way a straight line can’t capture. Polynomial relationships, where the outcome rises, peaks, and then falls (or vice versa), get flattened into a line that misses the turning point entirely. Saturation effects, where increases in the predictor produce smaller and smaller changes in the outcome, get overestimated at the extremes.
In some of these cases, you can transform your data (taking the logarithm of an exponential variable, for instance) to straighten the relationship and then apply linear regression. But if the underlying pattern is fundamentally non-linear, forcing a straight line through the data will give you coefficients that look precise but describe a relationship that doesn’t exist.
A subtler problem arises with multiple regression. When two or more predictor variables are highly correlated with each other, a situation called multicollinearity, the model struggles to separate their individual effects. A common diagnostic is the variance inflation factor (VIF). When the VIF for a variable exceeds 5 to 10, multicollinearity is likely distorting your results. The coefficients may swing wildly or flip signs, making the model unreliable even if the overall fit looks fine.
When Only Part of the Population Responds
One limitation that’s easy to overlook: linear regression can miss real effects when only a fraction of the population is actually affected. Research on methylmercury exposure illustrated this clearly. When Monte Carlo simulations modeled a scenario where only 5% of the population was sensitive to a toxin, linear regression failed to reliably detect the dose-response relationship in the overall sample, even with 700 participants. The estimated effect for the full population underestimated the true effect in the sensitive group by roughly tenfold. When the sensitive subgroup was 10% of the total, detection improved but still missed in some scenarios.
This matters in any field where a treatment or exposure affects a subgroup differently than the majority. The regression line reflects the average relationship across everyone, which can mask a strong signal hiding within a smaller group.
Simple vs. Multiple: Choosing the Right Version
Simple regression works well for exploratory analysis, when you want to see whether a single variable has any relationship with an outcome before building a more complex model. It’s also the right call when you genuinely have only one meaningful predictor, or when you want a highly interpretable result for communication purposes.
Multiple regression becomes necessary when you need to account for confounding variables. If you’re studying whether a new fertilizer improves crop yield, rainfall and soil quality also matter. Without including them, you might attribute their effects to the fertilizer. Multiple regression lets you isolate each variable’s contribution. The tradeoff is that each additional predictor requires more data to estimate reliably, and the model becomes harder to interpret as the number of variables grows.
The two versions serve different research goals as well. Etiological research, which asks whether a specific factor causes an outcome, tends to focus on one coefficient at a time, adjusted for confounders. Clinical prediction, which asks what outcome to expect for a patient with a particular profile, focuses on the combined prediction from all variables together. These different goals lead to different decisions about which variables to include and how many data points you need.