Perplexity
الحيرة
Definition
A measure of a language model's quality: the exponential of the average cross-entropy; the lower it is, the less “perplexed” the model is by the text.
Definition
Perplexity is the exponentiated average negative log-likelihood per token of a language model; lower values indicate better predictive fit.
Academic explanation
\(\mathrm{PPL}=\exp\!\big(-\frac1N\sum_i\log p(x_i\mid x_{<i})\big)\).
Practical examples
import math
p = [0.25, 0.25, 0.25, 0.25]
ppl = math.exp(-sum(math.log(x) for x in p) / len(p))
print(ppl)Expected output
4.0
Mathematical equations
\mathrm{PPL}=\exp\!\left(-\frac{1}{N}\sum_{i=1}^{N}\log p(x_i\mid x_{