What Is Model Distillation? How Large AI Models Teach Smaller Ones

Model distillation transfers behavior from a larger or more capable teacher into a smaller student model using teacher outputs, internal signals, or softened probability targets. This guide explains the mechanism, trade-offs, evaluation, and controls that matter in practice.