Mutual Information

المعلومات المتبادلة

Abbreviation: MI

Definition

How much one variable reveals about the other, including non-linear relationships.

Definition

Mutual information quantifies how much knowing one variable reduces uncertainty about another, capturing arbitrary dependence, and is zero if and only if they are independent.

Academic explanation

\(I(X;Y)=H(X)-H(X|Y)=\mathrm{KL}(p(x,y)\Vert p(x)p(y))\).

Learn and review

Suggested terms to learn next