Knowledge distillation trains a small student model to mimic a large teacher. Instead of only learning from hard labels, the student matches the teacher's softened probability distribution, which carries "dark knowledge" about which wrong answers are nearly right.
Unlock this and every premium problem across all tracks — plus hidden-test grading, advanced AI engineering modules and new content every week.
Start 7-day free trial