Softmax Function

دالة سوفت ماكس

Definition

A function that turns a vector of real numbers into a probability distribution summing to 1, used at the output of multi-class classification.

Definition

The softmax function maps a vector of real-valued scores to a probability distribution by exponentiating and normalising, and is widely used for multi-class outputs and attention weights.

Practical examples

Code example Python
import numpy as np

z = np.array([2.0, 1.0, 0.0])
e = np.exp(z - z.max())
print((e / e.sum()).round(3))

Expected output

[0.665 0.245 0.09 ]

Mathematical equations

\mathrm{softmax}(z)_i=\frac{e^{z_i}}{\sum_{j=1}^{K}e^{z_j}}

Learn and review

Suggested terms to learn next