Overfitting vs Underfitting: How to Spot and Fix Each

A model underfits when it is too simple to capture the pattern in the data, so it does badly on the training data and on new data. A model overfits when it learns the training data too closely, including its noise, so it does very well on the training data but badly on new data. The goal is the middle: a model that has learned the real pattern and nothing more.

The quickest way to tell them apart is to compare the error on the training data with the error on data the model has not seen:

	Training error	Test (validation) error	In bias-variance terms
Underfitting	High	High, similar to training	High bias
Good fit	Low	Low, close to training	Balanced
Overfitting	Very low	Much higher than training	High variance

See all three in one example

Delivery time grows with distance, but not in a straight line. This example fits a straight line (too simple), a curve (about right) and a very wiggly degree-15 curve (too flexible) to 21 training points, then checks each on 9 held-back points:

import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import PolynomialFeatures, StandardScaler
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_squared_error

# Delivery time (minutes) vs distance (km): a curved relationship plus noise.
rng = np.random.default_rng(3)
distance = rng.uniform(0, 10, 30).reshape(-1, 1)
minutes = 10 + 6 * distance.ravel() - 0.35 * distance.ravel() ** 2 + rng.normal(0, 2.5, 30)
X_train, X_test, y_train, y_test = train_test_split(distance, minutes, test_size=0.3, random_state=0)

print(f"{'degree':>6} {'train error':>12} {'test error':>11}")
for degree in (1, 2, 15):
    model = make_pipeline(StandardScaler(), PolynomialFeatures(degree), StandardScaler(), LinearRegression()).fit(X_train, y_train)
    train_err = mean_squared_error(y_train, model.predict(X_train))
    test_err = mean_squared_error(y_test, model.predict(X_test))
    print(f"{degree:>6} {train_err:>12.1f} {test_err:>11.1f}")

degree  train error  test error
     1         15.3        13.4
     2          5.7         6.7
    15          2.0       708.7

• Degree 1 underfits. A straight line cannot bend, so both errors are high. More data would not help; the model is simply too simple.
• Degree 2 fits well. Both errors are low and close together.
• Degree 15 overfits. It has the lowest training error, because it bends to pass near every training point, but its test error explodes: between and beyond the training points it swings wildly.

How to fix each

Underfitting (model too simple)

• Use a more flexible model, or add useful features (here: distance squared).
• Reduce regularisation if it is very strong.
• Train longer, for neural networks and boosted trees.
• Check the features actually contain the signal: no model can learn what the data does not show.

Overfitting (model too flexible for the data)

• Get more training data, if possible.
• Use a simpler model or fewer features.
• Add regularisation (L1, L2, dropout for neural networks).
• Limit tree depth, or stop training early when validation error starts rising.
• Use cross-validation to choose settings, instead of tuning on the test set.

Watch out for leakage

Sometimes a model looks perfect on both training and test data, then fails in production. That is often not a good fit but data leakage: information the model would not have at prediction time slipped into the features, or test rows leaked into training. The overfitting and data leakage lesson covers how to spot and prevent it.

Common mistakes

1. Judging a model by training error alone. It always rewards overfitting.
2. Tuning on the test set until it looks good, which overfits to the test set.
3. Adding complexity to fix overfitting, or more data to fix underfitting. Diagnose first with the training and validation errors.
4. One random split on a small dataset. Use cross-validation so the result does not depend on luck.

Interview angle

For “overfitting vs underfitting”:

• Define both in one sentence each, with the training-vs-test-error pattern.
• Connect them to bias and variance.
• Give two or three fixes for each.
• Mention cross-validation and data leakage.

Practise with the machine learning interview questions.

Key takeaway

Underfitting means the model is too simple: high error everywhere. Overfitting means it memorised the training data: low training error, high test error. Compare training and validation error to tell which you have, then make the model more flexible for underfitting, or simpler, regularised or better fed with data for overfitting.