Knowledge Distillation
تقطير المعرفة
Definition
Training a small model (the student) to imitate the outputs of a large model (the teacher), transferring its knowledge at a smaller size.
Definition
Knowledge distillation trains a compact student model to mimic the output distribution of a larger teacher model, transferring knowledge into a smaller network.
