Support Vector Machine
الة المتجهات الداعمة
Definition
A classification model that looks for the boundary with the largest possible margin between the two classes, and can separate non-linear data using a kernel.
Definition
A support vector machine is a supervised learning model that finds the maximum-margin hyperplane separating classes, and can learn non-linear boundaries using kernel functions.
Practical examples
from sklearn.datasets import make_moons
from sklearn.model_selection import train_test_split
from sklearn.svm import SVC
X, y = make_moons(n_samples=300, noise=0.2, random_state=0)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, random_state=0)
for k in ["linear", "rbf"]:
print(k, round(SVC(kernel=k, C=1.0).fit(X_tr, y_tr).score(X_te, y_te), 3))Expected output
linear 0.84 rbf 1.0
Mathematical equations
\min_{\mathbf{w},b,\boldsymbol\xi}\ \frac{1}{2}\lVert\mathbf{w}\rVert^2 + C\sum_{i=1}^{n}\xi_i \quad \text{s.t.}\quad y_i(\mathbf{w}^\top\mathbf{x}_i + b) \ge 1-\xi_i,\ \xi_i \ge 0Scientific sources
References and scientific sources
-
Research paperSupport-Vector Networks The foundational paper on the soft-margin support vector machine.
-
BookPattern Recognition and Machine Learning A classic reference on probabilistic machine learning.
-
BookThe Elements of Statistical Learning (2nd ed.) A statistical reference on machine learning, freely available from the authors' website.
