Batch Normalization

التطبيع الدفعي

Abbreviation: BN

Definition

A technique that normalizes a layer's outputs within each batch to zero mean and unit variance, then rescales them with learnable parameters.

Definition

Batch normalization normalises layer activations using the mini-batch mean and variance, then applies learnable scale and shift parameters to stabilise and accelerate training.

Mathematical equations

\hat{x}_i=\frac{x_i-\mu_B}{\sqrt{\sigma_B^2+\epsilon}},\quad y_i=\gamma\hat{x}_i+\beta

Learn and review

Suggested terms to learn next