Overfitting

الضبط الزائد (الملائمة الزائدة)

Definition

A situation in which the model memorizes the details and noise of the training data, so it performs well on them and poorly on new data.

Definition

Overfitting occurs when a model fits the training data, including its noise, so closely that it fails to generalise to unseen data; it shows as low training error but high validation or test error.

Practical examples

Code example Python
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier

X, y = load_breast_cancer(return_X_y=True)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.3, random_state=0)

for depth in [None, 3]:
    m = DecisionTreeClassifier(max_depth=depth, random_state=0).fit(X_tr, y_tr)
    print(depth, round(m.score(X_tr, y_tr), 3), round(m.score(X_te, y_te), 3))

Expected output

None 1.0 0.912
3 0.967 0.947

Mathematical equations

\mathbb{E}\big[(y-\hat f(x))^2\big] = \mathrm{Bias}\big[\hat f(x)\big]^2 + \mathrm{Var}\big[\hat f(x)\big] + \sigma^2

Scientific sources

References and scientific sources

Learn and review

Suggested terms to learn next