Random Forest

الغابة العشوائية

Abbreviation: RF

Definition

An ensemble model that trains a large number of decision trees on random samples and then combines their predictions by voting or averaging.

Definition

A random forest is an ensemble learning method that trains many decision trees on bootstrap samples with random feature subsets and aggregates their predictions by majority vote or averaging.

Practical examples

Code example Python
from sklearn.datasets import load_breast_cancer
from sklearn.ensemble import RandomForestClassifier

X, y = load_breast_cancer(return_X_y=True)
rf = RandomForestClassifier(n_estimators=200, oob_score=True, random_state=0).fit(X, y)
print(round(rf.oob_score_, 3))

Expected output

0.965

Mathematical equations

\mathrm{Var}\Big(\frac{1}{B}\sum_{b=1}^{B} T_b(x)\Big) = \rho\,\sigma^2 + \frac{1-\rho}{B}\,\sigma^2

Scientific sources

References and scientific sources

  • Research paper
    Random Forests Leo Breiman · 2001 · Machine Learning 45(1) The original random forest paper.
  • Book
    The Elements of Statistical Learning (2nd ed.) Trevor Hastie, Robert Tibshirani, Jerome Friedman · 2009 · Springer A statistical reference on machine learning, freely available from the authors' website.

Learn and review

Suggested terms to learn next