Distilling Earned Consensus from Delphi-Style Model Panels

When several language models deliberate over a hard question, they often reach better answers than any one of them alone, but at many times the cost. A natural fix is to train a single model on what the panel agreed, so the group's judgment becomes a fast, automatic skill. This proposal argues that the fix is unsafe as usually done. Panels frequently agree for bad reasons: they defer to the visible majority or stop arguing to end the discussion. A student trained on every agreement learns those errors too, and then repeats them quickly and with confidence. We propose to run the panel as a Delphi study. A Delphi study is a structured expert-consensus method: panelists first answer independently and anonymously, then see the group's answer distribution and anonymous rationales, and dissenters must either defend or revise their position. That structure makes it cheap to test offline, for each training question, whether a consensus was earned by argument or produced by conformity. Only earned consensus is compiled into the student. Questions where the panel merely conformed become training targets for a learned "convene the panel" decision.

The framing comes from human skill acquisition. People move from slow, effortful, deliberate processing to fast automatic execution through practice. Models of brain-computer-interface training (Towards a Cognitive Model of MI-BCI User Training) stress two further points. The resources that predict performance shift from general to task-specific as learning proceeds (Advances in user-training for mental-imagery-based BCI control). And a sizable fraction of learners, 10–30%, never reach automaticity at all. We borrow the second point as a design constraint: some kinds of question should never become automatic. A good compiled system knows which ones those are.

Core Claim

Consensus reached by a Delphi-structured panel can be split into two kinds. The first is earned consensus, which forms when panelists see rationales and survives an injected challenge. The second is conformist consensus, which forms just as readily when panelists see only the vote distribution, or collapses when challenged. We claim three things. First, a student trained only on earned consensus makes fewer confident errors than a student trained on all consensus. Second, the conformity label is a better signal for when to escalate back to the full panel than the student's own confidence or a prompt-based complexity check. Third, together these give a better accuracy-per-panel-call curve than existing routing and distillation methods. The key ingredient is the per-item label from counterfactual feedback. It is available only because Delphi separates feedback into a numeric part (the distribution) and an argumentative part (the rationales), and the two can be withheld independently.

Relation to Prior Work

The closest work is Self-Improvement of Language Models by Post-Training on Multi-Agent Debate (MACA). MACA runs multi-round debate among clones of one model and treats traces that agree with the final consensus as preferred. It then post-trains with preference learning and reports, for example, +16.3% on GPQA. MACA assumes the final consensus is a reliable winner label. We test that assumption directly, and we use a heterogeneous panel rather than clones, since asymmetric expertise is the defining feature of Delphi panels.

Two papers motivate the concern that consensus can mislead. Silence is Not Consensus finds that 61% of standard multi-agent clinical consensus goes uncontested, a pattern it calls silent agreement. Its fix is a dissent-injecting "catfish" agent that reduces this to 17% at inference time, and it reports a 12.73-point absolute gain over DeepSeek-R1 on MedAgentsBench. We reuse its dissent mechanism, but as an offline probe for labeling training data rather than as a cost paid on every query. Voting or Consensus? supplies two things: evidence that consensus beats voting on knowledge tasks (MMLU 54.0% vs. 51.7%), and a "drift" effect in which more rounds lower accuracy as agents concede. It also supplies the independent-drafting step we use as Delphi round one.

Two papers constrain the routing design. Adaptive Collaboration Strategy for LLMs in Medical Decision Making (MDAgents) routes queries by an LLM complexity check, which its authors find correlates poorly with clinician ratings. It still reaches strong accuracy with about 9.3 API calls. Single-agent or Multi-agent Systems? Why Not Both? documents that multi-agent systems consume 4–220x more prefill tokens and show an overthinking penalty on easy tasks. Both argue that escalation should be selective. Neither learns when to escalate from the panel's own behavior.

Three papers show that compilation can work at all, but each compiles a different teacher. NeSyPr compiles a symbolic planner into single-pass procedural memory and cuts latency by 90%. BARD and DAIL show that naive truncation and plain supervised fine-tuning compress reasoning poorly. None of the three compiles a group process, and none has to decide which teacher outputs deserve to be compiled.

Research Program

The panel. The teacher is a panel of five heterogeneous models, drawn from different model families and given different specialist role prompts, run as a machine Delphi.

In round one, each panelist answers independently and writes a rationale. In round two, each panelist sees the anonymized answer distribution and anonymized rationales. Panelists in the minority must either write a defense or revise. Round three repeats this. The panel stops early once a supermajority holds.

Labeling earned versus conformist consensus. Each training question gets two cheap counterfactual probes.

The first is a numbers-only rerun. Round two is re-executed with only the distribution shown and the rationales hidden. If panelists converge as much without the arguments as with them, the convergence is attributed to conformity.

The second is a challenge probe. A catfish-style dissenter, following Silence is Not Consensus, attacks the final consensus for one additional round.

An item's consensus is labeled earned when two conditions hold: the gain in convergence from rationales over numbers alone is positive, and the consensus survives the challenge. Otherwise it is labeled conformist. The label is binary for training, and we also keep the continuous gap for analysis.

This labeling triples the panel cost per training item. The cost is paid once, offline.

The student. The student is a single mid-sized open model, trained with preference learning in the style of MACA with two changes:

  • Earned items: preferred traces come only from items with earned consensus, and dissenting traces serve as the rejected side.
  • Conformist items: the target is an explicit defer output, so the student learns to hand these back to the panel.

At deployment the student answers directly unless its defer probability exceeds a threshold. The threshold is swept to trace a curve of accuracy against average panel calls per query.

Analysis. We will analyze which question types never compile, meaning they stay conformist or unstable across panels. We will also test whether this "never automatic" set is predictable from surface features, which is the analogue of predicting BCI non-learners.

Baselines

Naive reference points. The simplest reference is the untrained student answering alone, with no escalation. The second is the full Delphi panel run on every query, which gives the accuracy ceiling at maximum cost. All systems use the same student base model, the same training questions, and the same number of preference-training steps. Hyperparameters are tuned on a shared validation split with an equal number of trials per method.

MACA. The strongest distillation competitor is MACA (Self-Improvement of Language Models by Post-Training on Multi-Agent Debate). It will be reimplemented twice: once on its native clone debate and once on our heterogeneous Delphi panel's final consensus, with no conformity filter. The second version is the central ablation, because it isolates the effect of the earned/conformist label. It will be paired with confidence-based escalation, so that any gain cannot be attributed to having an escalation option at all. MACA reports +16.3% on GPQA over its base model. We beat it if our student has a lower confident-wrong rate at equal or better accuracy on GPQA and on the medical sets.

MDAgents. The strongest routing competitor is MDAgents (Adaptive Collaboration Strategy for LLMs in Medical Decision Making). It will be run with the same panel models as its multidisciplinary team. It reports a 5.2% improvement over the best single agent on DDXPlus at about 9.3 API calls. We beat it if our curve lies above it at matched calls per query.

Catfish at inference. The accuracy ceiling for structured deliberation is catfish-augmented deliberation at inference (Silence is Not Consensus), with its reported 12.73-point absolute gain over DeepSeek-R1 on MedAgentsBench. No reported absolute accuracy for that system was found on Lacuna. We therefore re-measure it on our panel. Success means recovering most of its accuracy at a fraction of its calls, not exceeding it.

Self-consistency with supervised fine-tuning. The last baseline is self-consistency voting followed by supervised fine-tuning on majority answers.

Why not just…?

Why not just run MACA on the panel's consensus? This is the real threat. If silent agreement mostly lands on correct answers, filtering adds little and MACA wins on data efficiency, because we discard items. The central ablation measures exactly this. If labeling earned consensus does not reduce confident errors on items where the teacher panel was wrong, and does not improve the accuracy-cost curve, the proposal's core claim fails. We will report that outcome as a negative result.

Why not just escalate on the student's own uncertainty? Students distilled from confident consensus may be confidently wrong precisely where the panel conformed. The drift result in Voting or Consensus? suggests that conformist agreement looks stable on the surface. We compare the learned defer head against maximum-probability and self-consistency-variance triggers at matched escalation rates. If uncertainty matches the defer head, the conformity label is unnecessary for routing, though it may still help training.

Why not just deploy the catfish panel on every query? It is likely the most accurate option. However, the token overhead documented in Single-agent or Multi-agent Systems? Why Not Both? makes it impractical at scale, and the same paper reports an overthinking penalty on easy items. Our accuracy-per-call curve shows directly how much of its accuracy survives compilation.

Why not just prompt for complexity, as MDAgents does? Its own authors report that the complexity check agrees poorly with clinicians. Our defer signal is grounded in observed panel dynamics rather than a prompt, and the head-to-head at matched calls settles which is better.

Proposed Experiments

Benchmarks. The evaluation uses the MedAgentsBench text sets (MedQA, PubMedQA, MedMCQA), DDXPlus, MMLU-Pro, GPQA, and SQuAD 2.0. SQuAD 2.0 is included because consensus is reported to handle unanswerable questions better than voting, which makes it a natural test of whether "no answer" survives compilation. Training questions come from the benchmarks' training splits, with test splits held out.

Metrics. Accuracy is reported as a function of average panel calls per query. We summarize each curve by its area over a fixed range of call budgets. The confident-wrong rate is measured on the subset where the full panel's consensus was incorrect, since that subset is where conformity leakage would show. The silent rate of the student's escalated cases is also reported.

Decisive checks. There are three.

The first check validates the conformity label. Conformist items should have lower panel accuracy than earned items. If they do not, the label means nothing.

The second check is the earned-only versus all-consensus ablation, at identical training steps.

The third is a transfer test. A defer head trained on the medical sets is applied to MMLU-Pro and GPQA, to see whether "which questions resist automaticity" generalizes across domains or has to be relearned for each one.