A group of models that make different mistakes on different demographic groups can be fairer together than any one of them alone. One model over-predicts a diagnosis for older patients, another under-predicts it, and their average cancels the error. That ensemble is expensive to deploy. The usual fix is knowledge distillation, which trains one small "student" model to imitate the ensemble's softened output probabilities. Distillation, however, is known to make students less fair than their teachers. A small model spends its limited capacity on the easy, high-agreement majority of the data and fits cheap shortcuts there. The inputs where the ensemble's fairness was actually produced, the ones where teachers disagreed and cancelled each other's biases, receive the least of the student's attention.
We propose to make teacher disagreement the training signal. The disagreement is measured as the part of the ensemble's uncertainty that comes from members contradicting each other, not from the label being inherently ambiguous. The student is then trained to spend more effort on exactly those inputs, and optionally to reproduce the spread of teacher opinions rather than only their average. This needs no demographic labels and no hand-built "biased" reference model. The claim is falsifiable: teacher disagreement locates the samples that matter for fairness better than the student's own loss or the ensemble's total uncertainty do. The approach should also fail, predictably, when the teachers agree on a shared shortcut. We will measure that boundary rather than hide it.
Core Claim
Let $p_1(x),\dots,p_M(x)$ be the class-probability outputs of $M$ teachers on input $x$, and let $\bar p(x)$ be their average. Write $H$ for entropy. The disagreement score is the ensemble's mutual information:
$$d(x) = H(\bar p(x)) - \frac{1}{M}\sum_{m=1}^{M} H(p_m(x)).$$
This score is near zero when teachers agree, even if they are all uncertain. It is large when individually confident teachers contradict each other. The student $s_\theta$ minimizes a reweighted distillation loss:
$$\mathcal{L}(\theta) = \frac{1}{\sum_i w_i}\sum_i w_i\, \mathrm{KL}\big(\bar p_T(x_i)\,\|\, s_{\theta,T}(x_i)\big), \qquad w_i = 1 + \lambda\, \frac{d(x_i)}{\mathbb{E}[d]}.$$
Here $T$ is the shared softening temperature, $\bar p_T$ and $s_{\theta,T}$ are the temperature-softened ensemble and student outputs, and $\lambda \ge 0$ sets how strongly disagreement is upweighted. $\lambda=0$ recovers ordinary ensemble distillation.
The central empirical claim is that in diverse teacher pools, $d(x)$ concentrates on minority and intersectional subgroups more than the student's per-sample loss or $H(\bar p)$ does. If so, disagreement weighting recovers the ensemble's fairness in the student with no group labels during training.
Relation to Prior Work
The ensemble side motivates the idea. Muffin combines frozen off-the-shelf networks under a trained aggregation head. It reports simultaneous fairness gains on ISIC2019 of 26.32% for age and 20.37% for disease site, but it keeps the full ensemble at inference. Bian & Zhang prove that the ensemble's discrimination is bounded through the teachers' pairwise bias correlations. Bias reduction therefore depends on members erring differently, which is precisely what $d(x)$ measures per input. Zafar et al. show that randomized combinations of models with opposing biases can meet fairness constraints that no single member meets. None of these three works compress the ensemble.
The distillation side constrains the design. What's Left After Distillation? finds that 10–41% of classes change accuracy significantly after distillation. It also finds that temperature helps fairness only up to a point before outputs collapse, so we treat $T$ as a tuned control and not as the mechanism. Ensemble Distribution Distillation shows that matching the averaged prediction discards teacher disagreement. It preserves that disagreement with a Dirichlet output head, but studies only calibration and out-of-distribution detection. We borrow its head as an optional component and test whether preserving disagreement adds anything beyond reweighting by it.
Four papers are the closest competitors, and each needs something we drop:
- AGRE-KD reweights teachers per batch by how far their gradients diverge from a deliberately shortcut-biased reference model. That requires knowing the shortcut in advance. AGRE-KD reweights teachers; we reweight samples from the teachers' own disagreement.
- MST-KD builds one teacher per demographic group, which requires demographic labels.
- FairDi also builds one teacher per demographic group and likewise requires demographic labels.
- Fairness without Demographics through KD shows that a single overfitted teacher implicitly upweights hard boundary samples without demographics. Its signal is per-sample difficulty, which mixes in label noise and inherently ambiguous inputs. Ours isolates contradiction between models, and only an ensemble can supply that.
AGRE-KD also reports something that contradicts our premise. Ensembles can agree on shortcuts, in which case $d(x)$ would be low on exactly the minority samples. We take that as the condition under which the method should fail, and we design a test for it.
Research Program
Teacher pools. The first component varies how diverse the teachers are, along three levels:
- Same architecture with different random seeds.
- Different architectures and pretraining sources, in the style of Muffin's ResNet/DenseNet/ShuffleNetV2 body.
- Pools that mix ERM-trained models with members trained by blind debiasing methods.
All teacher outputs are computed once, offline. The added cost over standard distillation is one entropy computation per sample.
Coverage diagnostic. The second component is a check we run before any student is trained. For each benchmark and pool, we take the top-$q$ quantile of training samples ranked by $d(x)$. We then measure what fraction of each annotated subgroup and each intersectional cell falls inside it, using group labels only for this analysis. We compare that fraction with the same ranking done by the student's loss and by $H(\bar p)$.
We predict that coverage tracks the student's gain in worst-group accuracy across pools and datasets. This turns the open question in the file, of when ensemble knowledge transfers shortcuts versus useful boundary information, into a measurable quantity. Low coverage should predict that our method fails, and also where AGRE-KD's reference model is genuinely needed.
Student variants. The third component compares three students: disagreement weighting alone, the Dirichlet head alone, and both together.
Multi-attribute analysis. On dermatology data, we track whether disagreement weighting avoids the seesaw effect, where improving fairness on one attribute worsens it on another. The concrete check is whether every annotated attribute's gap shrinks together. We expect this to work because the weighting is attribute-agnostic, and multi-attribute settings are where label-requiring methods scale worst.
Baselines
Do-nothing reference. This is an ERM student of the same small architecture trained on hard labels, alongside plain averaged-ensemble distillation ($\lambda=0$, the AVER setting in AGRE-KD). AGRE-KD reports that AVER reaches 46.4% worst-group accuracy (WGA) on Waterbirds. Our method must clearly exceed $\lambda=0$ with the same pool, temperature grid, and epochs. Otherwise disagreement contributes nothing.
Strongest competitor: AGRE-KD. It is the best reported ensemble-to-student method for worst-group accuracy, at 87.9% WGA on Waterbirds and 91.9% on CelebA, which exceeds its debiased teacher ensemble's 90.3% (AGRE-KD). We run it on identical teacher pools and student architectures with its biased reference model, under the same hyperparameter search budget and the same model-selection rule. We report two selection protocols, once with a group-labeled validation set and once with worst-class selection only. Matching AGRE-KD within a percentage point without a reference model counts as success. Exceeding it counts as a strong result. If AGRE-KD wins only on pools with low coverage, that confirms the predicted boundary rather than refuting the method.
Blind single-teacher baseline. This is Chai et al.'s overfitted-teacher distillation with linear logit mapping. It reports Equalized Odds of 11.59% versus 18.82% for the baseline on New Adult, and roughly 80.8% CelebA accuracy versus about 76.6% for DRO. We reproduce it on New Adult, COMPAS, and CelebA with a teacher whose compute matches the total compute of our pool.
Label-using baselines. On medical data, FairDi reports a mean minimum AUC of 0.9050 and an AUC gap of 0.0166 across five datasets. It also serves as the GroupDRO comparison. FairDi uses group labels, so it acts as a ceiling rather than a like-for-like competitor.
Multi-attribute fairness gains. For multi-attribute fairness, the reference point is Muffin's uncompressed ensemble on ISIC2019. For Fitzpatrick17k, no reported number for Muffin was found on Lacuna, so we will rerun it.
Ablations. These isolate the key ingredient. We replace $d(x)$ with three alternative weights, all at matched average weight:
- The student's own loss.
- The total entropy $H(\bar p)$.
- Random weights.
Why not just…?
Why not just upweight the samples the student gets wrong? Loss-based reweighting, as in the implicit mechanism Chai et al. identify, cannot separate genuinely ambiguous or mislabeled inputs from inputs where models disagree for systematic, group-linked reasons. Medical and toxicity labels are noisy, which makes this a real concern. The loss-weight ablation, combined with the coverage diagnostic, settles it directly. If loss weighting matches disagreement weighting, the ensemble adds nothing, and we will report that.
Why not just use the ensemble's total uncertainty? Total entropy rises on inherently hard images that every model finds unclear, and those need not be demographically skewed. Mutual information isolates contradiction. This is the cheapest serious threat, since it is a one-line change. The entropy-weight ablation decides it, and we expect the gap to be largest on pools built from different architectures.
Why not just use AGRE-KD, or train GroupDRO directly? AGRE-KD works well when the shortcut is known. It needs a reference model built for that shortcut, though, and it has only been shown on single-attribute benchmarks. GroupDRO needs group labels and pays an accuracy cost; on CelebA, DRO's roughly 76.6% accuracy compares with roughly 80.8% for KD in Chai et al. When teachers share a shortcut, AGRE-KD may really be better. The coverage diagnostic is designed to say in advance which regime a dataset falls into.
Why not just tune the temperature? The temperature study shows a single scalar temperature helps only within a narrow range and then collapses the outputs. Temperature also cannot focus effort on particular inputs. We tune $T$ for every method, so any remaining gain is attributable to the weighting.
Proposed Experiments
Spurious-correlation benchmarks. On Waterbirds, CelebA, and CivilComments, we follow AGRE-KD's setup with ResNet-50 and BERT-scale teachers and smaller students. We report worst-group and average accuracy across seeds, along with the coverage diagnostic for each pool.
Tabular benchmarks. On New Adult and COMPAS, we report Equalized Odds and accuracy against Chai et al. These sets also have discriminative risk measured by Bian & Zhang, which we compute for teachers and students to test whether the student keeps the ensemble's bound-level reduction.
Multi-attribute dermatology. On ISIC2019 and Fitzpatrick17k, we distill a Muffin-style off-the-shelf pool into a ShuffleNetV2 student. We report per-attribute and intersectional fairness gaps for age, site, and skin type, plus accuracy and inference cost relative to the full ensemble. Running on HAM10000 as well allows a direct comparison with FairDi's minimum AUC and AUC gap.
Decisive checks. Three checks determine the outcome:
- Disagreement weighting must beat both loss and entropy weighting on worst-group accuracy in diverse pools.
- Its advantage must shrink in seed-only pools.
- Across all runs, coverage must predict the student's worst-group improvement.
If the third check fails, then $d(x)$ is not doing what we claim, whatever the headline numbers show.
References
- Muffin: A Framework Toward Multi-Dimension AI Fairness by Uniting Off-the-Shelf Models
- Increasing Fairness via Combination with Learning Guarantees
- On Fairness, Diversity and Randomness in Algorithmic Decision Making
- What’s Left After Distillation? How Knowledge Transfer Impacts Fairness and Bias
- Ensemble Distribution Distillation
- Adaptive Group Robust Ensemble Knowledge Distillation
- MST-KD: Multiple Specialized Teachers Knowledge Distillation for Fair Face Recognition
- Fair Distillation: Teaching Fairness from Biased Teachers in Medical Imaging
- Fairness without Demographics through Knowledge Distillation