Paying for the Big Teacher Only When a Small Committee Disagrees

Many strong small models today are trained on labels produced by a much larger model. TrueTeacher, for example, ran a 540-billion-parameter model over 1.4 million document–summary pairs to teach a far smaller factual-consistency checker. Most of that expensive labeling is spent on easy examples that a handful of cheap models would label correctly anyway. We propose that a committee of small, open models label every synthetic example by soft voting: each model outputs a probability for each label, and the committee averages those probabilities instead of counting discrete votes. The expensive teacher is called only on the examples where the committee is unsure, or where its members disagree with one another. The student then trains on the committee's averaged probabilities where they are trusted and on the teacher's label elsewhere.

The bet is that disagreement between differently built models is a better signal of "this label needs the big model" than any single model's own confidence. If that holds, most of the teacher's cost can be removed with little loss in student quality. The proposal differs from a generic cascade in exactly this respect, and the experiments are built around testing it head-on.

Core Claim

On synthetic factual-consistency data, we claim that a calibrated soft-voting committee of models in the 1–8B range, plus a gate that escalates to the teacher, can do the following:

  • reach the student quality of full-teacher annotation while sending only a minority of examples to the teacher;
  • beat a gate that uses one small model's confidence at the same teacher budget;
  • beat a gate that uses the committee's hard majority vote at the same teacher budget.

For each example $x$, let $p_m(x)$ be member $m$'s calibrated label distribution, $w_m$ its performance-based weight, and $\bar p(x)=\sum_m w_m p_m(x)$ the soft vote. With $H$ denoting entropy, the gate score is

$$ g(x) = H(\bar p(x)) + \lambda \Big[ H(\bar p(x)) - \sum_m w_m H(p_m(x)) \Big]. $$

The first term measures how uncertain the averaged prediction is. The bracketed term measures how much the members disagree beyond their individual uncertainty. The weight $\lambda$ trades the two off. Examples are escalated to the teacher in decreasing order of $g$ until the teacher budget is spent. The disagreement term is the ingredient under test: setting $\lambda=0$ reduces the method to an ordinary confidence-based cascade.

Where This Sits in Prior Work

Work that motivates the proposal by documenting the cost. TrueTeacher (TrueTeacher) annotates 1.4M pairs with FLAN-PaLM 540B and distills them into a T5-11B student. That student reaches 87.8 average ROC-AUC on TRUE, against 84.9 for the teacher. The student beating its own teacher suggests that per-example teacher quality matters less than coverage, which is the slack we plan to exploit. Promptagator (Promptagator) and DISCO (DISCO) show the same pattern for generation followed by filtering: large models produce candidates, and a large share is thrown away after checking.

Work that already puts several models into the data pipeline. These are the closest neighbors.

  • Ensemble-Instruct (Ensemble-Instruct) generates full outputs from three 20–40B models and keeps those whose Rouge-L overlap agrees. Its filter is a hard, surface-level vote, and it triples generation cost.
  • UDAPDR (UDAPDR) distills several rerankers trained on synthetic subsets into one retriever. It aggregates after training rather than deciding, per example, where expensive labels are needed.
  • Committee voting for dataset distillation (CV-DD) weights architectures by proxy performance with a temperature softmax. It is confined to image distillation and never routes to a stronger model.

None of these uses committee disagreement to allocate a teacher budget. That allocation is the new piece here.

Work that supplies the method.

- Uncertainty-Aware Answer Selection scores candidates by length-normalized, teacher-forced log-likelihood across heterogeneous models and averages the scores. It reports roughly 4% over majority voting on GSM8K. We reuse this scoring to read label probabilities from generative members in one forward pass each, but apply it offline to dataset labeling rather than test-time answer selection. - ABCD at SemEval-2025 and CV-DD supply the performance-based member weights.

  • TRUE shows that heterogeneous checkers are complementary: ANLI scores 81.5, $Q^2$ 80.7, and their ensemble with SummaC 86.0 ROC-AUC. This is our evidence that members from different families will disagree informatively.

Work that constrains the design.

  • RL-Focal (Dynamic Optimizations of LLM Ensembles) shows that ensembles are worth their cost only when members fail on different examples. We therefore select the committee for low error correlation rather than raw accuracy, but avoid its learned routing agents, which need warm-start data.
  • TRUE also reports that checkers degrade sharply on grounding contexts beyond about 200 tokens. Long documents are therefore where we expect the committee to be confidently wrong, and the analysis must look there explicitly.

Research Program

Generating the data. We will rebuild a TrueTeacher-style pipeline with open models, because FLAN-PaLM 540B is not available to us. Summaries of varying quality are generated for documents. A teacher labels each one as consistent or inconsistent. A T5-family student is trained on the labels. The teacher stand-in is the largest open instruction-tuned model we can serve.

The full-annotation run with this stand-in is the upper reference. Our results are comparable to TrueTeacher's 87.8 in spirit, but not numerically, and we will say so.

Building the committee. The committee mixes model families:

  • an NLI classifier;
  • two or three small generative models from different families, scored by teacher-forced label likelihood;
  • optionally a question-generation-and-answering checker, if its cost fits.

Each member is temperature-scaled on a small held-out seed set of human-labeled pairs. The seed set is disjoint from every TRUE test split. Member weights come from seed-set ROC-AUC, using a temperature softmax as in CV-DD. Membership is chosen greedily to minimize pairwise error correlation on the seed set.

Using the labels. We sweep teacher budgets of 5, 10, 20 and 40 percent of examples. Escalated examples take the teacher's label. The others take $\bar p(x)$ as a soft training target. We will also test sharpening $\bar p(x)$ to a hard label, to separate the value of soft targets from the value of routing.

Accounting for cost. Cost is measured in forward-pass FLOPs and GPU-hours, with committee cost included. Running the committee on every example costs far less than one teacher pass, but it is never treated as free.

Transfer test. A second, smaller study carries the idea to Ensemble-Instruct's setting. One open generator produces candidate instruction responses. The committee then scores each candidate by teacher-forced likelihood and soft-votes to keep it or drop it. Ensemble-Instruct instead generates three times and checks Rouge-L agreement.

Baselines

Naive reference points. The first is the do-nothing student: the same T5 student trained only on the small human seed set. The second is the best off-the-shelf checker used directly with no distillation. TRUE reports 81.5 ROC-AUC for ANLI, and we will re-measure it on our splits.

Full-teacher annotation. This is the strongest existing method: TrueTeacher's recipe with our teacher stand-in labeling every example. TrueTeacher reports 87.8 average ROC-AUC on TRUE for its T5-11B student. No reported result was found on Lacuna for an open-teacher replication, so we will produce one.

We count the proposal as beating this reference if it reaches within 0.5 ROC-AUC of full-teacher annotation using at most 20 percent of the teacher's calls. It would also count if it exceeds the full-teacher student at equal total compute, with the saved compute spent on more synthetic examples.

Cheaper allocation strategies at matched teacher budget. Each is given identical data, an identical student, an identical hyperparameter search (same number of trials on the seed dev split), and an identical teacher-call budget. The strategies are:

  • random selection of which examples go to the teacher;
  • single-model confidence gating with the best committee member, run both with and without the rest of the committee providing non-escalated labels;
  • hard-majority-vote gating, which escalates on split votes;
  • our gate with $\lambda=0$.

The last of these is the ablation that isolates the disagreement term. A second ablation trains the student on hard committee labels instead of $\bar p$.

Transfer study. The baselines are Ensemble-Instruct's Rouge-L consensus filter and an unfiltered single-generator pool. Ensemble-Instruct reports 46.8 Rouge-L on SuperNI for MPT-7B, against 39.9 when MPT-7B is trained on Self-Instruct GPT-3 data. Our target is to match 46.8 at roughly one third of its generation compute.

Why not just…?

Why not just use the single best small model as the annotator? If one 8B model labels as well as the committee, the ensemble is wasted effort. TRUE's finding that ANLI alone scores 81.5 while a three-way ensemble scores 86.0 suggests single checkers leave real accuracy behind, but that comparison was on evaluation rather than labels for distillation. Our zero-teacher-budget point, comparing the committee soft vote with the best member alone, measures this directly.

Why not just escalate on one model's own confidence? This is the most serious threat. Confidence cascades are simple and often strong, and if $\lambda=0$ or a single-member gate matches our gate across budgets, the disagreement idea fails. Our reason to expect otherwise is that a single model is confidently wrong in systematic ways, and members from other families may not share those errors. We will report escalation precision, meaning the fraction of escalated examples where the teacher overturns the committee, for every gate. If single-model gating ties, we will report that as the finding.

Why not just have the teacher label fewer examples? TrueTeacher's student beating its teacher hints that coverage matters, so a smaller fully teacher-labeled set may lose more than the gated approach. That is a hypothesis, not a result. The random-allocation baseline and a "teacher labels only the budgeted subset, student trains only on that subset" run settle it.

Why not just filter with hard consensus, as Ensemble-Instruct does? Hard votes throw away margin information, which is the core argument for soft voting in Ensemble Methods for Unsupervised WSD. Consensus over fully generated text is also what triples Ensemble-Instruct's cost. The hard-vote gate and the transfer study test both points.

Proposed Experiments and Decisive Checks

Main experiment. We plot student ROC-AUC on each TRUE dataset and on its average against total labeling compute, for every allocation strategy and budget. The decisive check is whether our gate's curve lies above both the $\lambda=0$ and single-model-confidence curves at budgets of 20 percent and below, with three seeds and paired bootstrap intervals over test examples.

Label audit. For a random 2,000-example sample, we have the teacher label everything and measure three things: committee–teacher agreement; the rate of confident-wrong committee labels, which the gate cannot catch; and how both vary with document length. If confident-wrong errors cluster beyond about 200 tokens, as TRUE's findings predict, we test a length-aware rule that always escalates long documents.

Transfer study. We report SuperNI Rouge-L against generation-plus-filtering compute, comparing our filter with Ensemble-Instruct's.

What would falsify the claim. If the disagreement term adds nothing over plain confidence, or if the committee's correlated errors make the student worse than full-teacher annotation at every sub-full budget, the central claim is refuted.