Learn/Core Concept How does knowledge distillation work? Knowledge distillation trains a smaller student model to match a larger teacher's softened outputs, retaining useful behaviour at lower cost. It matters for efficient local models, as GPT-Sol illustrates. PruningQuantisation |