Normalization vs Standardization

When one column ranges from 0 to 1 and another from 0 to 1,000,000, many algorithms let the larger-scale column dominate simply because its numbers are bigger, not because it matters more. Feature scaling fixes that by putting columns on a comparable scale. The two standard methods are normalization (min-max scaling to a fixed range like 0 to 1) and standardization (z-score scaling to mean 0, standard deviation 1). They sound interchangeable but behave differently, and picking the right one depends on your data's shape and the algorithm you feed it.

This topic connects tightly to outlier detection and treatment, because outliers affect the two methods in very different ways, and it often runs alongside categorical data encoding when preparing features for a model.

Normalization (min-max scaling)

Normalization rescales a column into a fixed range, almost always 0 to 1, using the formula (x - min) / (max - min). The smallest value becomes 0, the largest becomes 1, everything else lands in between. It is intuitive and keeps all values bounded, which some algorithms require.

Its weakness is outliers: because it uses the min and max, a single extreme value stretches the range and crushes every normal value into a thin band near zero.

Standardization (z-score scaling)

Standardization rescales a column to have mean 0 and standard deviation 1, using (x - mean) / std. The result is a z-score: how many standard deviations each value sits from the mean. Values are not bounded to a fixed range, but they are centred and comparable across features. This is the assumption many linear and gradient-based models are happiest with, and it is less brutally distorted by a single outlier's presence than min-max is.

Worked example in pandas

Here is an illustrative sample with two very different scales, plus one outlier.

import pandas as pd

df = pd.DataFrame({
    "age":    [22, 25, 31, 40, 52],
    "income": [30000, 42000, 51000, 65000, 900000],  # last value is an outlier
})

# normalization: min-max to 0..1
df["age_norm"]    = (df["age"] - df["age"].min()) / (df["age"].max() - df["age"].min())
df["income_norm"] = (df["income"] - df["income"].min()) / (df["income"].max() - df["income"].min())

# standardization: z-score (ddof=0 for population std)
df["age_std"]    = (df["age"] - df["age"].mean()) / df["age"].std(ddof=0)
df["income_std"] = (df["income"] - df["income"].mean()) / df["income"].std(ddof=0)

print(df.round(3))

Expected output:

   age  income  age_norm  income_norm  age_std  income_std
0   22   30000     0.000        0.000   -1.101      -0.550
1   25   42000     0.100        0.014   -0.826      -0.514
2   31   51000     0.300        0.024   -0.275      -0.488
3   40   65000     0.600        0.040    0.550      -0.447
4   52  900000     1.000        1.000    1.651       1.999

Look at the normalized income: the outlier at 900,000 became 1.0, but it squashed the four genuine incomes into 0.000–0.040 — they are now nearly indistinguishable. Standardization spread them a little more (-0.550 to -0.447) while flagging the outlier as a large positive z-score of about 2.0. That contrast is the whole lesson: min-max is fragile to outliers, so treat outliers first or prefer standardization when they are present.

When to use which

• Normalize when you need bounded inputs, when the data has a known min and max, or for distance-based methods like k-nearest neighbours and k-means where a fixed range keeps features comparable.
• Standardize when the data is roughly bell-shaped, when comparing features that follow a normal-ish distribution, or for linear and gradient-based models that assume centred data.
• Skip scaling for tree-based models — decision trees, random forests and gradient-boosted trees split on thresholds and are indifferent to scale.

How analysts use it

Scaling is a modeling-prep step, not a general cleaning step, so analysts apply it only when the downstream method benefits. The habit is to check the algorithm first: distance- or gradient-based, scale; tree-based, do not bother. When scaling, they handle outliers beforehand, because an untreated extreme wrecks min-max and skews the mean and standard deviation that standardization relies on. A critical discipline in real pipelines is to compute the scaling parameters (min/max or mean/std) on the training data only and apply them to test data, so information from the test set does not leak into training.

Common mistakes

• Min-max scaling data with outliers. One extreme value dominates the range and compresses everything else. Treat outliers first or standardize instead.
• Scaling tree-based model inputs unnecessarily. It adds complexity for no benefit; trees ignore scale.
• Fitting the scaler on all data. Computing min/max or mean/std across train and test together leaks information. Fit on train, apply to test.
• Forgetting to scale at prediction time. New data must be transformed with the same parameters used in training, or predictions are meaningless.

In interviews

"What is the difference between normalization and standardization, and when would you use each?" is a frequent question. A strong answer gives both formulas in plain terms, ties normalization to bounded ranges and distance methods, ties standardization to normal-ish data and linear/gradient models, and adds two nuances: outliers hurt min-max more, and tree models need no scaling. Mentioning that scalers must be fit on training data only shows you understand data leakage, which sets you apart.

Where this fits in your learning path

Normalization versus standardization is a bridge topic between cleaning and modeling in the data cleaning cluster. It depends on outlier detection and treatment, since outliers change which method is safe, and it complements categorical data encoding in feature preparation. Understanding feature scaling is a solid step on the data analyst roadmap.