Mutual Information
المعلومات المتبادلة
Definition
How much one variable reveals about the other, including non-linear relationships.
Definition
Mutual information quantifies how much knowing one variable reduces uncertainty about another, capturing arbitrary dependence, and is zero if and only if they are independent.
Academic explanation
\(I(X;Y)=H(X)-H(X|Y)=\mathrm{KL}(p(x,y)\Vert p(x)p(y))\).
