Entropy

الإنتروبيا

Definition

A measure of uncertainty in a probability distribution, and the average information needed to describe a random outcome.

Definition

Entropy measures the uncertainty of a probability distribution, equal to the expected information content of an outcome.

Practical examples

Code example Python
import numpy as np

def H(p):
    p = np.array(p)
    return float(-(p * np.log2(p)).sum())

print(H([0.5, 0.5]), round(H([0.9, 0.1]), 3))

Expected output

1.0 0.469

Mathematical equations

H(p)=-\sum_{i}p_i\log_2 p_i

Learn and review

Suggested terms to learn next