Entropy
الإنتروبيا
Definition
A measure of uncertainty in a probability distribution, and the average information needed to describe a random outcome.
Definition
Entropy measures the uncertainty of a probability distribution, equal to the expected information content of an outcome.
Practical examples
import numpy as np
def H(p):
p = np.array(p)
return float(-(p * np.log2(p)).sum())
print(H([0.5, 0.5]), round(H([0.9, 0.1]), 3))Expected output
1.0 0.469
Mathematical equations
H(p)=-\sum_{i}p_i\log_2 p_i