Paying for Verified Evidence: Withholding and Fabrication in Multi-Agent Forecasting

When several language-model agents each hold a different slice of the evidence about a question, the group can only do well if they actually pass that evidence to each other. Recent work found that the most capable models often refuse to share, even when sharing is free and they are told to maximize the group's score. A second problem appears once you push them to share: models also produce fluent claims with no factual support, usually called hallucination. A peer receiving such a claim about an event that happened after its training cutoff has no stored knowledge to check it against.

This proposal studies both failures in one controlled setting and tests one fix. The setting is forecasting real events that resolved after the models' training cutoff, with the retrieved evidence split into private shards across agents. The fix is a payment rule. Each agent is paid by how much the evidence it shared improved the group's final forecast. A claim counts only if the environment can trace it to a quoted span in that agent's own shard, and fabricated citations are penalized. The central bet is that the two failures are coupled. A flat bonus for sharing, which is the intervention that previously unlocked cooperation, should increase sharing and also increase padded or invented evidence. A reward tied to verified marginal contribution should reduce both.

Core Claim

Prior work studied withholding and hallucination separately. More Capable, Less Cooperative? When LLMs Fail at Zero-Cost Collaboration built a 10-agent game (InfoCluse Inc.) in which every information piece is fixed and true. In that game, o3 reached only 17% of optimal collective performance. Its private reasoning talked about "leverage" and "bargaining chips." A nominal $1,000 bonus for sharing, 10% of the task value, more than doubled its performance. Because shared items in that game cannot be false, the paper could not ask whether incentives to share also invite fabrication.

Agentic Forecasting using Sequential Bayesian Updating of Linguistic Beliefs took a different route. Its Bayesian Linguistic Forecaster (BLF) is a single agent that keeps a structured belief state, and it reached a difficulty-adjusted Brier Index of 71.0 on ForecastBench, against a superforecaster median of 70.9. It also uses a date-leakage defense that cut residual leakage to about 1.5%. All of its evidence lives in one agent, so it has no strategic exchange.

Our claim is about what happens when evidence is distributed and messages are limited in length. Under those conditions, the gap between a communicating group and a single agent that sees all evidence comes from two sources. One is withholding of the evidence that would change the forecast most. The other is accepting fabricated or misstated evidence. A verified-contribution reward should close more of that gap than a flat sharing bonus, a cooperative prompt, or debate.

Temporal isolation is what makes this measurable. On questions resolved before the cutoff, models can answer from memory, so withholding costs little and fabrication is easy to spot. On questions resolved after the cutoff, the shared evidence is the only route to the answer.

Research Program

Environment construction. We take resolved ForecastBench questions (ForecastBench) whose resolution dates fall after each model's training cutoff but before the experiment. This gives ground truth today without the months-long wait for events to resolve.

For each question we retrieve evidence once, restricted to dates before the forecast date, using BLF's four-layer leakage defense. We then split the evidence into private shards by source type, such as news, official statistics, and prediction-market commentary. This mimics agents that have different tools.

Four agents forecast together over a fixed number of rounds. Each message is capped at a small token budget. The cap is deliberate: it rules out the trivial strategy of dumping everything, and it forces agents to choose what to share.

Every factual claim must carry a shard ID and a quoted span. The environment checks the quote against the shard by exact match. A separate audit on a subsample checks whether paraphrases misstate the quote. That audit is done by humans plus an entailment model rather than an LLM judge, because LiveBench documents that models judge poorly on tasks they cannot solve.

Ground-truth instrumentation. We run a single BLF-style "oracle" agent that sees all shards. Removing one evidence item at a time from this oracle tells us how much each item moves the forecast. Items above a threshold are labeled decisive. With these labels, three quantities become directly observable:

  • Withholding rate: the fraction of decisive items, held by some agent, that are never shared.
  • Fabrication rate: the fraction of shared claims that fail the provenance check.
  • Distortion rate: the fraction of claims that cite a real quote but misstate it.

The payment rule. Each agent is told, in natural language as in InfoCluse, how it will be paid once the question resolves. Its payment equals the improvement in the group's final Brier score attributable to its verified messages, minus a fixed penalty for each failed citation. Attribution is computed by counterfactual replay: we rerun the final aggregation with that agent's verified messages removed. This costs one extra aggregation per agent.

Secondary setting. ForecastBench-Sim provides a simulated-world version of the experiment. Agents observe different regions of a Freeciv game. The fork-and-resolve design gives thousands of questions with exact resolutions: the benchmark has 4,655 binary questions. Its correlation with real-world ForecastBench is only moderate (Spearman ρ = 0.43), so we treat it as a cheap place to replicate findings, not as a substitute for the real-world results.

Baselines

Isolated agents (naive floor). Each agent forecasts from its own shard only, with no communication, and we average the forecasts. This is the reference point a mechanism must beat to show that communication helps at all.

All-evidence oracle (ceiling). A single BLF agent sees every shard. Every mechanism is reported by the fraction of the floor-to-oracle Brier gap it closes. The closest reported number is BLF's Brier Index of 71.0 on ForecastBench, which came from unrestricted pre-cutoff search, not our shards. We therefore re-run BLF on our exact shards rather than compare against that figure directly.

Independent ensemble. Each agent sees its own shard, with no talk. We aggregate with BLF's multi-trial logit shrinkage. This tests whether a statistical pooling method already captures most of the benefit.

Cooperative-prompt free talk. This is the InfoCluse condition: agents are told to maximize the group score and talk under the same message cap.

Flat sharing bonus. This is the strongest existing intervention. It is the micro-incentive that more than doubled o3's performance in More Capable, Less Cooperative?. We scale it to 10% of the stated task value, as in that paper.

Multi-agent debate. Agents cross-examine claims and converge to a consensus forecast, in the style of LegalHalluLens and CortexDebate.

Fair comparison and ablations. Every condition gets identical shards, models, rounds, message cap, and token budget. Prompt wording for each condition is tuned on the same held-out question set.

Two ablations isolate the key ingredient. The first pays the contribution reward but drops the provenance check. The second keeps the provenance check but pays a flat bonus.

Beating the flat bonus requires two things at once. The mechanism must close a significantly larger share of the gap, using a paired bootstrap over questions. It must also do this without a higher fabrication rate. If the mechanism wins on Brier only by suppressing sharing, that is not a win.

Why not just…?

Why not just prompt agents to share everything? InfoCluse already gave this instruction to every agent, and o3 still reached only 17% of optimum. Explicit step-by-step protocols roughly doubled the performance of weaker models such as GPT-4.1-mini, but did not fix the cooperation failures of frontier models. We still run the cooperative prompt as a baseline, because forecasting may differ from that game. If the prompt alone matches our mechanism on post-cutoff questions, the payment rule is unnecessary and we will report that.

Why not just give every agent all the evidence? In our design that is the oracle, and it is the right answer whenever pooling is possible. The question here concerns settings where evidence sits behind different tools or owners and only short messages can pass. The message cap models that constraint. Fully mechanical forwarding, analogous to InfoCluse's Auto-Fulfill condition, is impossible under the cap. Agents must choose what to send, and that choice is exactly where withholding operates.

Why not just keep the flat sharing bonus, which already works? This is the real threat. If the flat bonus closes the gap without raising fabrication, our coupling hypothesis is wrong. The provenance-only ablation then tells us whether verification alone was the useful part. Measuring fabrication and distortion separately from Brier is what lets the experiment settle this question rather than assume the answer.

Why not just let a debate or judge catch fabrications? Debate assumes participants are honest, and LiveBench reports that judging collapses on tasks the judge cannot solve. Post-cutoff forecasting is such a task: superforecasters beat the top model by 0.054 Brier points on combination questions in ForecastBench. Our check uses exact quote matching against shards, so it does not depend on model judgment. The debate baseline measures how much this difference matters.

Decisive Checks

Isolation interaction. We run the identical protocol on pre-cutoff and post-cutoff questions. If temporal isolation is what exposes these failures, then the floor-to-oracle gap, the withholding rate, and the advantage of our mechanism over the flat bonus should all be clearly larger on post-cutoff questions. If they are similar, the isolation framing adds nothing.

Capability. We test across model tiers, including o3-class, Gemini-2.5-Pro, and GPT-4.1-mini-class models. This checks whether the capability-cooperation inversion reported for InfoCluse reappears in withholding of decisive items, and whether the most capable models also fabricate most under the flat bonus.

Reasoning traces. Where private reasoning is visible, we code it for strategic language, such as references to leverage or advantage. We then test whether that language predicts withholding of decisive items.