Support Vector Machine

الة المتجهات الداعمة

Abbreviation: SVM

Definition

A classification model that looks for the boundary with the largest possible margin between the two classes, and can separate non-linear data using a kernel.

Definition

A support vector machine is a supervised learning model that finds the maximum-margin hyperplane separating classes, and can learn non-linear boundaries using kernel functions.

Practical examples

Code example Python
from sklearn.datasets import make_moons
from sklearn.model_selection import train_test_split
from sklearn.svm import SVC

X, y = make_moons(n_samples=300, noise=0.2, random_state=0)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, random_state=0)
for k in ["linear", "rbf"]:
    print(k, round(SVC(kernel=k, C=1.0).fit(X_tr, y_tr).score(X_te, y_te), 3))

Expected output

linear 0.84
rbf 1.0

Mathematical equations

\min_{\mathbf{w},b,\boldsymbol\xi}\ \frac{1}{2}\lVert\mathbf{w}\rVert^2 + C\sum_{i=1}^{n}\xi_i \quad \text{s.t.}\quad y_i(\mathbf{w}^\top\mathbf{x}_i + b) \ge 1-\xi_i,\ \xi_i \ge 0

Scientific sources

References and scientific sources

  • Research paper
    Support-Vector Networks Corinna Cortes, Vladimir Vapnik · 1995 · Machine Learning 20 The foundational paper on the soft-margin support vector machine.
  • Book
    Pattern Recognition and Machine Learning Christopher M. Bishop · 2006 · Springer A classic reference on probabilistic machine learning.
  • Book
    The Elements of Statistical Learning (2nd ed.) Trevor Hastie, Robert Tibshirani, Jerome Friedman · 2009 · Springer A statistical reference on machine learning, freely available from the authors' website.

Learn and review

Suggested terms to learn next