Counting the Shared Prior Once: Debiasing LLM Panels by Separating Prior from Evidence

When several language models vote or debate, their agreement is treated as independent confirmation. Much of that agreement is not evidence, though. The models were trained on overlapping web text with similar recipes, so they start from nearly the same beliefs before they see anything about the task. Pooling their answers counts this shared starting belief once per model. The result is a panel that sounds more certain than it should, and that sides with the obvious answer even when the clues held by individual members point elsewhere. Game theorists have a name for the assumption these aggregators silently make: the common prior assumption, the idea that agents differ only in their private information and not in their baseline beliefs. LLM panels break this assumption in an unusual way. Their priors are neither shared nor independent; they are heavily correlated.

We propose a simple, measurable fix. For each model, ask the question twice: once with its private evidence (a retrieved passage, a hidden clue, a news snippet) and once without it. The "without" answer distribution is that model's prior. The shift between the two is what its evidence actually contributed. Aggregate the shifts across models, and add the prior back only once, weighted by how many effectively independent priors the panel contains, which we estimate from how correlated the models' errors are on blind questions. The claim is that this one change removes a specific, predictable source of panel overconfidence and of the common-information bias in hidden-profile tasks. It also explains a puzzle in recent work on peer-prediction scoring for LLMs.

Core Claim

Write $p_i(a \mid q, e_i)$ for model $i$'s probability of answer $a$ to question $q$ given its private evidence $e_i$, and $p_i(a \mid q)$ for the same model's probability with the evidence withheld. With $N$ models, our aggregate is

$$\log P(a) = \frac{N_{\text{eff}}}{N}\sum_{i=1}^{N} \log p_i(a \mid q) \;+\; \sum_{i=1}^{N} w_i\left[\log p_i(a \mid q, e_i) - \log p_i(a \mid q)\right] + \text{const},$$

where $w_i$ is a per-model weight on its evidence-driven update, and $N_{\text{eff}}$ is the effective number of independent priors in the panel. If private evidence is conditionally independent given the true answer and priors are exact, Bayes' rule gives $w_i = 1$ and a prior counted once. Naive log-probability pooling, which is the implicit target of confidence-weighted voting, instead sets $N_{\text{eff}} = N$ and counts the prior $N$ times. We estimate $N_{\text{eff}}$ from blind items using the standard design-effect formula

$$N_{\text{eff}} = \frac{N}{1 + (N-1)\bar\rho},$$

where $\bar\rho$ is the average pairwise correlation between models' errors when none of them has private evidence. The formula comes from the variance of a mean of equicorrelated estimates.

The falsifiable consequence is specific. A panel's overconfidence on shared-information items should grow roughly with $N/N_{\text{eff}}$. Correcting the prior term alone should remove most of it without hurting accuracy, because the evidence term, which carries the panel's real information, is left intact.

This sits squarely on a gap identified in Bayesian Truth Serum for LLMs?. That paper applied a peer-prediction rule, which scores reports against forecasts of what peers will say, to panels of GPT-5 variants. It found 21–32% lower calibration error than majority vote, but significantly lower accuracy (0.9724 versus 0.9793). The authors attributed the failure partly to shared pre-training breaking conditional independence, but they did not correct for it. Our mechanism is a direct correction for that diagnosis, and it needs no assumption that LLMs respond strategically to scoring rules, which the same paper shows they do not.

Research Program

The theoretical starting point is the classical literature on relaxing the common prior. The Complexity of Agreement shows that communication and computation are not what blocks consensus among Bayesian agents; differing priors are. Incentives for Subjective Evaluations with Private Beliefs proves that no mechanism asking only for a rating can elicit truth from agents with heterogeneous priors, and its workaround is to infer priors from many related tasks. That result constrains our design: we cannot recover priors from answers alone. We sidestep the impossibility because LLMs, unlike humans, can be queried counterfactually with the evidence removed. This gives the prior directly rather than inferring it.

Elicitation for Aggregation shows that optimal pooling needs each agent's evidence precision, not just its point forecast. Our evidence term is a crude, observable proxy for this: a passage that barely moves a model's answer carries little weight. False Consensus, Information Theory, and Prediction Markets constrains where the method can work. Summing updates is correct when signals are substitutes, meaning independent given the state, and can fail badly when signals are complements, such as clues that only reveal the answer jointly. We treat that boundary as a test, not a footnote.

The system we would build has three parts. The first is elicitation. For each question, each model gives an answer distribution with and without its private evidence. For open-weight models we read option probabilities from the logits. For API models we use the frequency of answers over $k$ samples at a fixed temperature, with $k$ set by a pilot on held-out items. The no-evidence pass is shared across all agents that receive the same question. The cost is therefore one extra pass per model per question, not one per evidence item. This contrasts with LEAP, which makes a separate likelihood-elicitation call for every evidence piece.

The second part is prior-correlation estimation. On a few hundred blind items per domain, we compute each model's error indicator and the pairwise correlations, and from these $N_{\text{eff}}$. Each model's blind distribution also gets a single temperature fit on the same split. The third part is aggregation by the display equation above. It comes in two variants: theory-fixed ($w_i = 1$, $N_{\text{eff}}$ from correlations) and lightly fitted ($w_i$ and the prior coefficient fit on at most 500 calibration items). The 500-item cap borrows the scale that Uncertainty Calibration for Tool-Using Language Agents and Large Language Models Must Be Taught to Know What They Don't Know found sufficient for calibration.

The method is closest in spirit to two lines of work that decompose belief into prior and update. LEAP separates a base-rate prior from per-evidence likelihoods inside one model, and finds that conjugate aggregation roughly halves calibration error (0.184 to 0.087 on a diagnostic split). Agentic Forecasting using Sequential Bayesian Updating of Linguistic Beliefs shrinks divergent rollouts toward a base rate. Both are single-agent. Both ask the model to state likelihoods or summaries rather than measuring the update counterfactually. Neither models the fact that several agents' priors are correlated copies of each other, which is the quantity we estimate.

Three panel designs use different models but treat their outputs as independent votes. ReConcile mixes model families and rescales verbal confidence. Emergence of Biased Consensus in Multi-Agent LLM Debates shows that homogeneous debating groups snap into biased consensus, and that heterogeneity in model family or temperature breaks this. Consilience controls who speaks when on hidden-profile tasks. Our contribution relative to all three is a quantity, $N_{\text{eff}}$, that says how much heterogeneity a given panel actually has, and an aggregator that uses it.

The evaluation regimes come from the file. HiddenBench (65 hidden-profile tasks) and the Synthetic Travel Benchmark from Consilience give the setting where private evidence exists and common-information bias is the failure mode. ForecastBench gives a substitutes-like regime: each agent receives a different retrieved news bundle, and the prior is the no-news forecast. Natural Questions with retrieved passages split across agents, following the setup of When Do LLMs Need Retrieval Augmentation?, gives a factual-QA analogue. HotpotQA, with its two supporting paragraphs given to different agents, is a deliberately complementary regime where we predict failure. The MMLU/TruthfulQA/ARC probe style from the BTS paper gives the pure shared-information regime, where the evidence term is zero and only the prior correction acts. Panels span three settings: a homogeneous panel of one model at several seeds, a same-family panel at different sizes, and a cross-family panel mixing Llama-3.1, Qwen3, Mistral and one API model. These settings give a wide range of $N_{\text{eff}}$.

Baselines

The naive reference point is unweighted majority vote, or self-consistency (Wang et al.), over the same panel with each agent seeing its private evidence. It is the bar every panel method must clear on accuracy. In the BTS study it beat every peer-prediction variant on accuracy (0.9793 versus 0.9724 for BTS-E). We would run it with the same number of model calls as our method, which means giving majority vote the extra samples our blind pass consumes. Beating it means equal or higher accuracy with clearly lower expected calibration error and Brier score.

The strongest calibration competitor in the panel setting is the peer-prediction family from Bayesian Truth Serum for LLMs?, specifically BTS-E and Surprisingly Popular, implemented with that paper's prompt for forecasting the population's verdicts. The reported numbers are a 21–32% reduction in binned calibration error relative to majority vote, at a significant accuracy cost. To beat it we must match or exceed that calibration gain without the accuracy drop. That is the exact tension the paper left open.

On hidden-profile tasks the strongest existing system is Consilience. It reports up to 0.57 absolute accuracy over round-robin exchange and 0.13 over a full-information baseline averaged across 12 models. The file gives no absolute HiddenBench accuracy for Consilience. We would rerun it on our panels with its released controller calibration protocol and the same calibration-trajectory budget as our fitted variant. Our method is a one-shot aggregator, not a dialogue controller, so we also test the two composed, with our aggregator replacing Consilience's final vote. A fair win would be matching Consilience's accuracy at a fraction of its rounds, or improving it when composed.

On ForecastBench the strongest reference is BLF, with a difficulty-adjusted Brier Index of 71.0 against a superforecaster median of 70.9; a Claude 3.5 Sonnet baseline scores Brier 0.122 there. We would not claim to beat BLF's full agentic loop. Instead we would replace BLF's multi-trial averaging step with our prior-corrected pooling across differently-informed agents, holding retrieval fixed, and measure the Brier change.

Confidence-weighted voting with ReConcile's rescaling table, and LEAP run as a single model given all agents' evidence, complete the external set. The file reports no ReConcile result on our hidden-profile or forecasting benchmarks. For LEAP the only reported figure is the 0.184 to 0.087 calibration-error drop on its own diagnostic split.

Two ablations isolate the key ingredient. The first sets $N_{\text{eff}} = N$, which is plain log pooling of evidence-conditioned answers, and so tests whether counting the prior once is what matters. The second sets the evidence term to the full posterior rather than the posterior-minus-prior shift, and so tests whether the counterfactual blind pass earns its cost.

Why not just…?

Why not just temperature-scale the pooled panel output? This is the most serious threat. One fitted temperature on summed log-probabilities can shrink an overconfident panel, and Platt-style recalibration does a great deal of work in BLF. The difference is structural. A global temperature shrinks the prior and the evidence by the same factor, while our method shrinks only the prior. The test is to stratify items by how much private evidence agents hold, from none up to several passages. A single temperature fit on the pooled data should be too conservative on evidence-rich items and too confident on evidence-free ones. Our aggregator should be calibrated across all strata. If one temperature calibrates every stratum equally well, the prior/evidence separation is unnecessary, and we would say so.

Why not just hand all the evidence to one strong model? The full-information baseline is natural, and LEAP shows that structuring evidence inside one model helps. But Consilience reports that structured multi-agent deliberation beats full-information prompting by 0.13 on hidden-profile tasks, and LEAP itself documents context collapse when evidence is dumped into one prompt. We include full-information prompting and single-model LEAP as baselines. If either matches our panel at equal total tokens, the panel framing loses its justification on that benchmark.

Why not just make the panel more diverse? Okawa et al. and ReConcile both find that mixing model families helps. Diversity raises $N_{\text{eff}}$, but it does not tell the aggregator how far to trust agreement. Even cross-family LLMs share large amounts of pre-training data. Our correlation measurement predicts how much residual overconfidence a diverse panel still has. The experiment varies panel composition and checks whether the prior correction still helps once diversity is maximal. A shrinking but nonzero gain would show the two ideas are complementary.

Why not just probe hidden states for confidence? Probes such as PROBECAL reach very low calibration error on single models (0.690 to 0.035 on TabMWP). But they only read what one model already believes. They cannot combine clues spread across agents, and they need white-box access, which excludes API members of a cross-provider panel. Where weights are available, we use probe-calibrated probabilities as inputs to our aggregator, so the comparison is about pooling rather than single-model calibration.

Decisive Experiments

The first check is the scaling prediction on shared-information items. Across homogeneous, same-family and cross-family panels on the MMLU/TruthfulQA/ARC-style probe, we plot naive-pooling overconfidence, measured as mean confidence minus accuracy, against the measured $N/N_{\text{eff}}$. The claim predicts a roughly monotone relation, and predicts that the prior correction removes most of the gap while leaving majority-vote accuracy unchanged. No relation would falsify the mechanism.

The second check is the stratified-evidence comparison against a globally temperature-scaled pool on Natural Questions and ForecastBench, described above. It settles whether separating prior from evidence matters beyond simple shrinkage.

The third check is HiddenBench and Synthetic Travel. There the prediction is that subtracting each agent's prior removes the pull of the shared intuitive answer, because the shared clues move every agent's prior identically and are counted once. This should raise accuracy over majority vote and confidence-weighted voting, and should be competitive with Consilience at far fewer rounds.

The fourth check is the planned failure case. On HotpotQA with supporting paragraphs split across agents, Kong and Schoenebeck's complements analysis predicts that individual updates are near zero while the joint evidence is decisive, so additive pooling should fail. We would report this failure honestly. We would then test the minimal remedy their theory suggests: one round in which agents exchange evidence summaries before the evidence-conditioned pass. This tells us whether the method's boundary is where the theory says it is.

All results report accuracy, binned expected calibration error, Brier score and calls per item. Significance is assessed with Benjamini-Hochberg correction across benchmarks, matching the BTS study's protocol.