Quantization

التكميم

Definition

Reducing the numeric precision of weights and activations (for example from 32 bits to 8 or 4 bits) to make the model smaller and faster.

Definition

Quantization lowers the numeric precision of weights and activations, such as from 32-bit floats to 8- or 4-bit integers, shrinking model size and speeding inference.

Practical examples

Code example Python
import numpy as np

x = np.array([-1.0, 0.0, 0.5, 1.0])
s = np.abs(x).max() / 127
q = np.round(x / s).astype(int)
print(q.tolist())

Expected output

[-127, 0, 64, 127]

Learn and review

Suggested terms to learn next