Standardizing a variable means transforming its values so the data has a mean of 0 and a standard deviation of 1. You do this by taking each value, subtracting the mean of the dataset, and dividing by the standard deviation. The result is called a z-score, and it tells you how far each data point sits from the average in a universal unit of measurement.
How the Formula Works
The calculation is straightforward. For any single data point, subtract the overall mean of the variable, then divide by the standard deviation. If you have a test score of 85 in a class where the mean is 75 and the standard deviation is 5, the standardized value is (85 − 75) ÷ 5 = 2.0. That tells you the score is exactly 2 standard deviations above average.
After you apply this to every value in a dataset, the transformed variable will always center on zero. Values above the original mean become positive, values below it become negative, and values right at the mean become zero. The spread of the data gets rescaled so that one unit now equals one standard deviation. The shape of the distribution stays the same. If the original data was skewed, the standardized version will still be skewed. Standardization changes the scale, not the underlying pattern.
Why You’d Want to Do This
The most common reason is comparison. Imagine you scored 720 on one exam and 28 on another. Those raw numbers are meaningless side by side because the two tests use completely different scales. But if you standardize both scores, you can see which one placed you further above (or below) the average for that particular test. A z-score of 1.5 on the first exam and 2.1 on the second tells you clearly: your performance on the second test was relatively stronger.
This same logic applies in data analysis and machine learning. When a dataset contains variables measured in different units (height in centimeters, weight in kilograms, income in dollars), some variables will naturally have much larger numbers than others. Many algorithms interpret larger values as more important, which distorts results. Standardizing puts every variable on the same footing so no single one dominates just because of its scale.
Specific techniques where standardization matters include principal component analysis, k-means clustering, and any algorithm that relies on distance calculations between data points. Linear regression can work without standardization, but standardizing the inputs makes the coefficients directly comparable, which is useful when you want to know which predictor has the strongest influence.
Standardization vs. Normalization
These two terms get mixed up constantly. Standardization rescales data to have a mean of 0 and standard deviation of 1. Normalization typically rescales data to a fixed range, most often 0 to 1, by subtracting the minimum value and dividing by the range. The choice between them depends on context.
Standardization works well when your data roughly follows a bell curve or when outliers are present but you don’t want to compress most of your data into a tiny range. Normalization is a better fit when you need bounded values (for example, pixel intensities in image processing) or when the data doesn’t have a meaningful bell-shaped distribution. In practice, you can try both and see which gives better results for your specific analysis.
What Z-Scores Actually Tell You
Once a variable is standardized, the z-scores carry intuitive meaning. A z-score of 0 means the value equals the mean. A z-score of 1 means one standard deviation above average. A z-score of −2.5 means two and a half standard deviations below average.
If the data is normally distributed, these z-scores map to specific percentiles. About 68% of values fall between −1 and +1, roughly 95% fall between −2 and +2, and 99.7% fall between −3 and +3. A z-score beyond ±3 is rare enough that it often flags an outlier worth investigating. These thresholds come from properties of the normal distribution and won’t apply as neatly to heavily skewed data, but they’re a reliable rule of thumb in many real-world datasets.
Z-scores are also the basis for many statistical tests. When you calculate a test statistic and compare it to a critical value, you’re often working in standardized units. The entire z-table (or standard normal table) is built on this concept: it converts z-scores to probabilities, letting you answer questions like “what percentage of the population scores higher than this value?”
When Standardization Isn’t Necessary
Not every situation calls for it. Decision trees and random forests, for example, split data based on thresholds within individual features. They don’t compare distances across variables, so the scale of each variable doesn’t affect the outcome. Standardizing before using tree-based models won’t hurt, but it won’t help either.
You’d also skip standardization when the original units carry meaning you want to preserve. If you’re reporting results to an audience that understands the raw scale (doctors interpreting blood pressure in mmHg, for instance), converting to z-scores would strip away that interpretability.
Another consideration is what data you use to calculate the mean and standard deviation. If you’re building a predictive model, you compute these values from your training data only, then apply the same transformation to new data using those same training statistics. Recalculating from new data would shift your scale and break the model’s assumptions.
A Quick Example
Suppose five students scored 60, 70, 75, 80, and 90 on an exam. The mean is 75 and the standard deviation is roughly 10.6. After standardizing:
- 60 becomes (60 − 75) ÷ 10.6 = −1.41
- 70 becomes (70 − 75) ÷ 10.6 = −0.47
- 75 becomes (75 − 75) ÷ 10.6 = 0.00
- 80 becomes (80 − 75) ÷ 10.6 = 0.47
- 90 becomes (90 − 75) ÷ 10.6 = 1.41
The student who scored 75 sits exactly at zero. The student who scored 90 is 1.41 standard deviations above the mean. If you calculated the mean and standard deviation of these new z-scores, you’d get exactly 0 and 1. That’s the whole point: you’ve translated raw performance into a relative position on a common scale.