Perplexity

الحيرة

Abbreviation: PPL

Definition

A measure of a language model's quality: the exponential of the average cross-entropy; the lower it is, the less “perplexed” the model is by the text.

Definition

Perplexity is the exponentiated average negative log-likelihood per token of a language model; lower values indicate better predictive fit.

Academic explanation

\(\mathrm{PPL}=\exp\!\big(-\frac1N\sum_i\log p(x_i\mid x_{<i})\big)\).

Practical examples

Code example Python
import math

p = [0.25, 0.25, 0.25, 0.25]
ppl = math.exp(-sum(math.log(x) for x in p) / len(p))
print(ppl)

Expected output

4.0

Mathematical equations

\mathrm{PPL}=\exp\!\left(-\frac{1}{N}\sum_{i=1}^{N}\log p(x_i\mid x_{

Learn and review

Suggested terms to learn next