Softmax Function
دالة سوفت ماكس
Definition
A function that turns a vector of real numbers into a probability distribution summing to 1, used at the output of multi-class classification.
Definition
The softmax function maps a vector of real-valued scores to a probability distribution by exponentiating and normalising, and is widely used for multi-class outputs and attention weights.
Practical examples
import numpy as np
z = np.array([2.0, 1.0, 0.0])
e = np.exp(z - z.max())
print((e / e.sum()).round(3))Expected output
[0.665 0.245 0.09 ]
Mathematical equations
\mathrm{softmax}(z)_i=\frac{e^{z_i}}{\sum_{j=1}^{K}e^{z_j}}