Teacher's pet: understanding and mitigating biases in distillation

Michal Lukasik, Srinadh Bhojanapalli, Aditya Krishna Menon, Sanjiv Kumar

TL;DR

Knowledge distillation (KD) is a go-to technique for boosting a "student" model's performance using a "teacher," but this paper reveals a hidden cost: KD often harms performance on underrepresented or "hard" subgroups. The authors identify that students inherit and amplify the teacher's confident mispredictions and propose two adaptive methods—AdaAlpha and AdaMargin—to ensure distillation gains are distributed more equitably across all data classes.

What Problem Does This Paper Address?

In the world of deep learning, knowledge distillation is essentially a shortcut to better models. You take a big, complex "teacher" model and use its predictions to help train a smaller, more efficient "student." While this almost always improves average accuracy, the machine learning community has largely ignored a critical question: Does distillation help every subgroup of data equally?

The authors argue that distillation is not a "free lunch." In many cases, the student becomes a "teacher's pet"—it mimics the teacher so closely that it inherits the teacher's biases. If the teacher struggles with a rare class (e.g., a specific breed of dog with few training samples) or a sensitive demographic subgroup, the student might actually perform worse than if it had just been trained on the original labels alone. This creates a significant tension between achieving high aggregate performance and maintaining fairness or robustness on the "long tail" of data.

Methods

The paper investigates the mechanics of this bias through both empirical ablations and a theoretical framework.

  1. Theoretical Analysis: Using kernel ridge regression, the authors formally prove that distillation acts as a form of regularization. Crucially, this regularization disproportionately penalizes "low variance" directions in the data—which, in plain English, means the rarest subgroups suffer the most.
  2. AdaAlpha (Adaptive Mixing): Standard distillation uses a fixed weight ($\alpha$) to balance the teacher's advice with the ground-truth labels. The authors propose making this weight class-dependent. If a holdout set shows the teacher is unreliable or overconfident on a specific class, the student "mutes" the teacher for that class ($\alpha_y \to 0$) and relies more on the original labels.
  3. AdaMargin (Adaptive Margins): Borrowing from long-tail learning literature, this method introduces per-class margins into the loss function. It essentially increases the "penalty" for the student when it misclassifies hard classes that the teacher finds difficult to model, forcing the student to pay more attention to those subgroups.

Datasets

The authors validated their findings across a diverse range of benchmarks:

  • Image Classification: CIFAR-10, CIFAR-100, and ImageNet.
  • Long-Tailed Variants: CIFAR-100-LT and ImageNet-LT (where class imbalance is artificially exaggerated).
  • Fairness: The UCI Adult dataset, where subgroups are defined by sensitive attributes like race and sex.
  • Natural Long-Tail: The iNaturalist 2018 dataset, which is inherently imbalanced.

Key Insights and Ideas

  • The "Confident Misprediction" Trap: The most surprising finding is that teachers are often confidently wrong on hard classes. Because the student is trained to match the teacher's distribution, it inherits this misplaced confidence, leading to a "negative margin" where the student is certain about the wrong answer.
  • Error Amplification: Distillation doesn't just pass on errors; it can amplify them. In several experiments, the distilled student performed significantly worse on the "worst-k" classes than a baseline student trained without a teacher.
  • Fairness is a Distillation Issue: On the UCI Adult dataset, the authors found that distillation helped the majority group (high-income males) but significantly degraded performance for minority groups (e.g., Black women). This highlights that KD can be a hidden source of algorithmic bias.
  • KD as a Filter: Theoretically, the paper shows that KD behaves like a filter that suppresses signals from rare subgroups. While this helps "smooth" the model and improve average performance, it effectively erases the nuances needed to handle rare cases.

Key Results

Accuracy improvement for Average and Worst subgroups across Baseline and Adaptive methods.
  • ImageNet Baseline: Standard self-distillation improved average accuracy by +0.39%, but the accuracy on the 10 hardest classes dropped by -0.43%.
  • Long-Tail Impact: On CIFAR-100-LT, distillation boosted average accuracy by +2.17% but slashed the worst-subgroup accuracy by -1.46%.
  • Mitigation Success: The proposed AdaAlpha (Adaptive mixing) and AdaMargin (Adaptive margin) methods successfully mitigated these drops. As shown in the figure above, while the baseline harms the worst subgroups, the adaptive methods restore and even improve their performance without sacrificing average gains. For instance, on CIFAR-100, AdaMargin improved the "Worst-1" class accuracy to 51.33%, compared to just 49.00% for standard distillation and 45.67% for the one-hot baseline.

Limitations and Future Work

The primary limitation of the proposed adaptive methods is the requirement for a labeled holdout set. To calculate the per-class weights ($\alpha_y$), you need to know how the teacher performs on a "dev" slice of data. This might not be feasible in semi-supervised settings where distillation is used to label massive amounts of unlabeled data.

Future work could explore "unsupervised" ways to estimate teacher reliability or investigate whether these biases persist in more modern distillation setups, such as those involving feature-based distillation or large-scale foundation models.

Summary updated

Abstract

Knowledge distillation is widely used as a means of improving the performance of a relatively simple “student” model using the predictions from a complex “teacher” model. Several works have shown that distillation significantly boosts the student’s overall performance; however, are these gains uniform across all data sub-groups? In this paper, we show that distillation can harm performance on certain subgroups, e.g., classes with few associated samples, compared to the vanilla student trained using the one-hot labels. We trace this behavior to errors made by the teacher distribution being transferred to and amplified by the student model. To mitigate this problem, we present techniques which soften the teacher influence for subgroups where it is less reliable. Experiments on several image classification benchmarks show that these modifications of distillation maintain boost in overall accuracy, while additionally ensuring improvement in subgroup performance.
BibTeX
@inproceedings{lukasik2022teacher,
  author = {Michal Lukasik and Srinadh Bhojanapalli and Aditya Krishna Menon and Sanjiv Kumar},
  title = {{Teacher's pet: understanding and mitigating biases in distillation}},
  booktitle = {International Conference on Learning Representations},
  year = {2022},
  url = {https://openreview.net/forum?id=WDBo7y8lcJm},
}
Figure