One Forward Pass for a Verifier Panel: Distilling Verifier Ensembles While Keeping Their Disagreements

Many strong reasoning systems now work as a pipeline. A language model writes several candidate answers, a panel of separate "verifier" models (models that judge whether an answer is correct) scores each candidate, and an aggregator combines those scores to pick a winner. FUSE shows that this aggregation can be done without any labeled data. It treats the verifiers as noisy voters, estimates each one's reliability from how often they agree with one another, and weights them accordingly. With a panel of 7 to 33 verifiers it reaches 64.4% on GPQA Diamond. The price is a second pipeline stage that runs dozens of models on every candidate for every query, which the FUSE authors themselves call prohibitive for production serving.

We propose to replace that panel with one network: a shared backbone with one small output head per original verifier. It is trained to reproduce each verifier's verdicts and also the pattern of agreements and disagreements among them. The point of keeping separate heads is not ranking accuracy on the training distribution, where a single scalar score copied from FUSE might do just as well. The point is that FUSE's label-free reliability estimation can be rerun on a new domain at the cost of one forward pass. A collapsed scalar score freezes the verifier weights that suited the training data. A panel of heads that disagrees in the right way can be reweighted when verifier reliabilities shift.

Core Claim

The claim has two parts. First, a multi-head student can compress a heterogeneous verifier panel into a single pass while preserving enough of the panel's joint error structure for FUSE's unsupervised estimation to remain valid. "Joint error structure" here means the approximate conditional independence between verifiers, given the true label, that FUSE's method-of-moments step relies on. Second, this preserved structure is what makes the distilled system robust under domain shift. The decisive comparison is therefore a domain-transfer test: train the student on one set of tasks, then evaluate on held-out tasks where the original verifiers' relative reliabilities change.

The obvious alternative design would be to fold the verifiers into the generator itself. The research file flags the central risk of any shared backbone. Shared parameters push the heads toward correlated errors. That violates the triplet conditional independence assumption that FUSE uses, and its method-of-moments estimates could then fail silently. Our training objective targets this failure directly. Alongside matching each head to its teacher verifier, we match the covariance between heads within each pseudo-label class to the covariance of the teacher verifiers:

$$ \mathcal{L} = \sum_{j} \mathrm{BCE}\big(h_j(x,y),\, v_j(x,y)\big) + \lambda \sum_{c \in \{0,1\}} \sum_{j<k} \Big(\mathrm{Cov}_c(h_j,h_k) - \mathrm{Cov}_c(v_j,v_k)\Big)^2 $$

The symbols are as follows. $x$ is the prompt and $y$ a candidate answer. $v_j$ is the binarized verdict of teacher verifier $j$ after FUSE's score transformation, and $h_j$ is the student's head $j$. $c$ is the FUSE pseudo-label of the candidate. $\mathrm{Cov}_c$ is the covariance within a batch, computed only over candidates with pseudo-label $c$. $\lambda$ trades off fitting each verifier against reproducing how the verifiers depend on one another. We condition on the pseudo-label because the independence assumption is stated conditional on the true label, and the pseudo-label is the best unsupervised stand-in available.

Relation to Prior Work

Three lines of work collapse a test-time ensemble into a single pass, and each discards something we need to keep. BOND distills best-of-N sampling into the generator policy using a Jeffreys-divergence objective. It assumes one scalar reward model and removes the verification stage altogether, so nothing remains that could be re-estimated on a new domain. STEVE-2 distills a hierarchical multi-agent pipeline into one multimodal model, cutting inference from 91.44s to 9.22s. It too keeps only the final behavior and not the teachers' individual votes. Classical ensemble distillation is a constraint rather than a method source here. Improving Ensemble Distillation With Weight Averaging and Diversifying Perturbation shows that matching the ensemble mean erases diversity: on CIFAR-100, standard distillation has an NLL of 1.038, versus 0.792 when diversity is preserved through low-rank subnetworks. We take that finding as motivation for preserving per-member structure, but our target is a statistic that a downstream estimator consumes, not calibration.

The multi-head design itself is borrowed from Collaborative Learning for Deep Neural Networks. That paper trains several heads on a shared ResNet-50 at 1.5x the memory of a single model rather than 4x, and discards them at test time. We keep the heads at test time because their disagreement is the output. Joint Generator-Ranker Learning trains a generator and a ranker jointly but keeps both stages at inference. Its authors list collapsing them as future work, and one arm of our study tests exactly that.

On the systems side, Understanding and Optimizing Multi-Stage AI Inference Pipelines motivates why removing a stage matters beyond FLOPs: scoring and retrieval stages tend to dominate tail latency, and every stage boundary adds serialization and glue-code cost.

Research Program

We first reproduce a FUSE pipeline using the panel and generators from that paper. From it we log per-verifier transformed verdicts and pseudo-labels for many candidates drawn from source tasks, namely MATH500 and MMLU-Pro. We then train students in two placements.

In the standalone placement, a single mid-sized backbone carries one head per verifier. Each candidate costs one pass instead of up to 33.

In the internalized placement, the heads read the generator's own hidden states. These states already exist once a candidate has been written, so verification becomes nearly free. The two placements test whether a generator can judge its own outputs with the needed independence. We expect the internalized heads to inherit the generator's blind spots and to show higher conditional correlation. If they do not, the pipeline collapses entirely to a single model.

For every student we measure three things on held-out data: per-head agreement with its teacher verifier, the size of conditional-independence violations among heads compared with the original panel, and FUSE's estimated sensitivity and specificity computed from heads versus from teachers. At test time on target tasks (GPQA Diamond and Humanity's Last Exam), we rerun FUSE's unsupervised estimation on the heads' outputs, with no labels, and select answers.

Baselines

The naive reference points have no label-free reweighting at all. One is majority vote over candidates; the other is the unweighted average of the full verifier panel. FUSE reports improving on naive ensembling and majority vote by 10 to 20 percentage points on MMLU-Pro. We rerun both with our generator and candidate pool so that the comparison is internal and controlled.

The strongest existing method, and our ceiling, is full FUSE with its whole panel. The paper reports 64.4% accuracy on GPQA Diamond with Llama 3.3 70B, compared with 64.1% for the WEAVER baseline using 5% labels, and 54.3% on HLE. Our student succeeds if it comes within a small margin of this ceiling on GPQA Diamond and HLE while spending a stated, measured fraction of the panel's verifier compute and peak memory. The same generator, the same N candidates per prompt, and the same transformed teacher verdicts are used throughout.

The baseline that most directly threatens the proposal is a scalar distilled verifier. It uses the same backbone and the same data and compute, but is trained to regress FUSE's final aggregated score, which is the BOND-style "collapse to one reward" choice. It receives an equal hyperparameter-search budget. If it matches the multi-head student on target domains, our central claim fails.

The ablation that isolates our key ingredient sets $\lambda = 0$, giving per-head distillation with no covariance matching. A second ablation trains heads that are independent and lightly regularized, which tests whether the covariance term specifically, rather than having heads at all, matters. We also compare against speculative rejection applied to the full panel. It reports up to 5x speedup and under 1% score loss on AlpacaFarm-Eval, but it reduces generation cost rather than verifier cost. We report the two combined to show the savings are complementary.

Why not just…?

Why not just distill FUSE's final score into one reward model? This is the real threat, and on the source domain we expect it to tie. Its weakness is structural. FUSE's verifier weights are estimated separately for each dataset, and a scalar student bakes in the source-domain weights. The transfer experiment settles the question directly: train both students on MATH500 and MMLU-Pro, then evaluate on GPQA Diamond and HLE. If the scalar student holds up there, we would report that per-head structure is unnecessary, which is itself a useful negative result.

Why not just use the single best verifier? On HLE, FUSE reports that the oracle best verifier reaches 53.5%, close to FUSE's 54.3%, so the ensemble's margin can be thin. Choosing that oracle, however, requires labels, and which verifier is best changes across tasks. We report the best single verifier selected on source labels and evaluated on target tasks. This makes visible how much of the ensemble's value is simply finding the right verifier without labels.

Why not just speed up the existing pipeline with early rejection? Speculative rejection prunes candidates early but still loads and queries every reward model, and its thresholds are fixed across prompts. It leaves the panel's memory footprint and the stage boundary in place, which is what the MIST study identifies as a driver of tail latency. We treat it as complementary and measure the two combined.

Why not merge the verifiers' weights? The panel mixes architectures and training runs that sit in different regions of parameter space. Weight averaging only works within a single connected low-loss region, so merging is not applicable here.

Decisive Checks

The proposal stands or falls on three measurements. The first is whether FUSE's estimates computed from student heads track the estimates from the real panel on target domains. The second is whether conditional-independence violations among heads stay near the panel's level when covariance matching is on and grow when it is off. The third is whether, on GPQA Diamond and HLE, the multi-head student with label-free reweighting beats the scalar distilled verifier while approaching full FUSE's 64.4% and 54.3%. Alongside these we report verifier FLOPs per candidate, peak GPU memory, and median and 99th-percentile wall-clock latency for every system. If the internalized arm passes all three checks, generator-plus-panel reduces to a single model. If only the standalone arm passes, the result is still a two-stage pipeline in which the second stage is one network instead of dozens.