Research · Hugging Face Blog ·
Making Knowledge Distillation Cheap Enough to Run at Scale
Hugging Face presents methods for reducing the cost of knowledge distillation, a process for transferring capabilities from larger models to smaller ones. The post focuses on making distillation practical for large-scale model training and deployment.