Knowledge Distillation

تقطير المعرفة

Definition

Training a small model (the student) to imitate the outputs of a large model (the teacher), transferring its knowledge at a smaller size.

Definition

Knowledge distillation trains a compact student model to mimic the output distribution of a larger teacher model, transferring knowledge into a smaller network.

Learn and review

Suggested terms to learn next