Most real misinformation is not a flat lie. It strings accurate facts into a false impression: a statistic quoted without its denominator, a real photo given a new caption, a true event framed as a trend. Fact-checkers grade such claims on scales like "Mostly True" or "Partly True," but automated systems do poorly on them. Two kinds of systems have been tried. Single-model classifiers learn surface cues from the claim text. Debate systems have language models argue for and against a claim and then let another model, acting as a judge, read the whole argument. That judge is swayed by polished prose. This note proposes a different split of labor. Adversarial debaters do not get to win the case. Their only job is to surface short, checkable propositions, especially the unstated implications and the omitted context that make a true-sounding claim misleading. A separate scorer then checks each proposition against retrieved evidence and never sees the debaters' rhetoric. It reads off the model's probability of answering "Yes" to the question "Does this evidence support this statement?"
The graded verdict is then computed from the pattern of scores. A claim whose stated facts are supported but whose implied conclusion is refuted lands in the middle of the scale for an explicit reason, not because a judge found one side more eloquent. In two sentences: debate is good at finding what a claim leaves out, and token probabilities are good at checking a single narrow statement against evidence. So we use debate only to generate the statements and let probabilities do the judging, which removes the fluency bias that undermines text-based judges.
Core Claim
On fine-grained veracity benchmarks, we expect a pipeline that scores debate-surfaced propositions with probabilities, without letting the scorer read the arguments, to beat two alternatives. The first is a judge model that reads the full debate transcript. The second is direct probability scoring of the claim alone. The gain should come mostly from intermediate classes (Mostly-True, Partly-True, Mostly-False) and from claims published after the models' training cutoff.
The mechanism makes two testable predictions. First, most of the gain over claim-level scoring comes from implied and context propositions, not from the claim's literal content. Second, rewriting the debaters' arguments in more persuasive prose should move a transcript-reading judge but leave our scores nearly unchanged.
What Exists and Where This Departs
Three systems apply debate to misinformation, and all three end in a text-generating judge. Debate-to-Detect runs a five-stage debate between affirmative and negative agents, followed by a judge panel that scores factuality, source reliability, and other dimensions in text. Its ablation shows the judging stage matters most, with a 3.61–5.10 F1 drop when removed. That result motivates us, because it locates the leverage in adjudication, which is exactly what we replace. MADISSE forces opposing initial stances and shows this catches subtle inconsistencies a single model misses. We reuse that stance assignment, but its adjudicator still reads the transcript and emits a discrete label. MADR uses debate to refine fact-check explanations. Its gap between a 4.99/5 automated judge score and 30% human-rated faithfulness is our sharpest evidence that transcript-reading judges reward fluency. MAD-Sherlock brings debate and web retrieval to out-of-context images but treats the task as binary.
The scoring method comes from text-to-image evaluation. VQAScore shows that the probability of "Yes" to a yes/no verification question tracks human judgment far better than embedding similarity. TIFA shows that breaking a prompt into atomic questions makes such scores interpretable. We borrow both, with one change. TIFA decomposes only what the text literally says, and in misleading claims the problem is precisely what the text does not say. Debate is our tool for producing that missing material.
Three papers use probabilities around debate but for other purposes. LASE uses token log-probabilities to decide whether to run a debate at all. ConfMAD passes calibrated confidences between debaters. Both work on math and science QA, and both use probability as a control signal rather than as the verdict. LASE also constrains our design: it reports that overconfident leaders never trigger debate. We predict this failure is worst on misleading claims, whose literal content is confidently true. DiVA produces continuous factuality scores, but it does so with a trained regression head over a feedforward search pipeline, with no adversarial step and only literal claims. Distributional LLM-as-a-Judge trains judges to match the spread of human labels. That is complementary, and it is a candidate for the final aggregation step.
Research Program
For each claim $c$ we retrieve one shared evidence pool $E$, so every compared method sees identical evidence. Two debaters with forced opposite stances then run a short, fixed-round exchange. They are not asked who is right. They are asked to state, as standalone declarative sentences, what $c$ literally asserts, what a typical reader would conclude from it, and what relevant fact it omits. Each sentence is tagged with one of these three roles.
A deduplication pass merges near-identical propositions. A separate open-weight scorer model (Qwen-2.5 or Llama-3, chosen because proprietary APIs hide full token probabilities) then sees only $E$ and one proposition $p_i$. It computes
$$s_i = \frac{P(\text{Yes} \mid E, p_i)}{P(\text{Yes} \mid E, p_i) + P(\text{No} \mid E, p_i)},$$
where the probabilities are the scorer's next-token probabilities after the question "Does the evidence support this statement? Answer yes or no." We also compute a parallel "Does the evidence contradict it?" score. This lets the scorer separate "unsupported" from "refuted," which matters for the Unverifiable class and for the "not enough information" midpoint of the VeriTaS scale. Both scores are temperature-scaled on a development split, following the calibration practice in ConfMAD.
The graded verdict is an ordinal mapping from a handful of role-wise summary features. These are the minimum support among literal propositions, the mean support among implied propositions, and the maximum refutation among context propositions. The mapping is fit as ordinal logistic regression with fewer than twenty parameters. We keep it this small deliberately: the claim is that the propositions carry the signal, not a learned model. An untrained fixed-weight version is reported alongside it. For VeriTaS, the same features map to the continuous scale in [-1, 1]. The media properties of VeriTaS use the identical construction with a vision-language scorer, in the style of VQAScore, applied to image–caption propositions. We treat this media variant as an extension and report it separately.
Baselines
The naive reference points are the majority class and the claim-only model. On X-Fact, the claim-only model reaches 38.2% macro F1 in-domain, and any method that fails to clear it is exploiting style rather than evidence.
The strongest fine-tuned prior system on X-Fact is that paper's attention-based evidence aggregator with metadata. It reports 41.9% macro F1 in-domain and roughly 15–16% on out-of-domain and unseen-language splits. Beating it requires exceeding 41.9 in-domain while improving the out-of-domain figure by a clear margin, which is where an evidence-grounded, non-fine-tuned method should shine.
On VeriTaS, the strongest reported systems are search-augmented frontier models. Gemini 3 Pro reports MSE 0.39 and GPT-5.2 reports 0.45. Our open-weight scorer starts at a capability disadvantage, so we report our pipeline in two forms. The first uses open-weight models end to end. The second uses a frontier model as debater while keeping the open-weight scorer. Matching 0.39 with the open scorer would be a strong result, and the more important target is a smaller rise in error on the post-cutoff longitudinal split than the reported jump from about 0.6 to above 0.8.
The strongest debate competitor is Debate-to-Detect. It reports 81.97% F1 on Weibo21 against 74.97% for standard debate. We found no reported result on Lacuna for Debate-to-Detect on X-Fact or VeriTaS, so we reimplement it with the same backbone, the same evidence pool, and a matched token budget. We also run it on Weibo21 to check our reimplementation against the published number.
The ablation that isolates our key ingredient keeps our debate and propositions fixed and swaps only the judging step. The variants are a transcript-reading text judge with the D2D rubric, the same judge's probability over verdict tokens, and our rhetoric-blind proposition scoring. A second ablation drops implied and context propositions and scores only literal ones. This is TIFA-style decomposition applied to claims, and it tests whether debate-surfaced material is what helps. Every method gets the same number of hyperparameter trials on the development split.
Why not just…?
Why not just score the claim directly with a probability, no debate? A single "Is this claim supported by the evidence?" probability is cheap and already continuous, and it is the most serious threat. Our expectation is that it scores Partly-True and Mostly-True claims as true, because the evidence does support their literal content. The literal-only ablation and the claim-level baseline test exactly this. If neither loses meaningfully to the full pipeline on the intermediate classes, the proposal's central premise is wrong and we will say so.
Why not just have one model list the implications and missing context? Adversarial debate is not obviously needed to generate propositions; a single prompted model, or several samples from it pooled together, might enumerate implications just as well. MADISSE's evidence that forced opposing stances surface inconsistencies a single evaluator overlooks suggests debate helps, but that evidence comes from summaries, not claims. We therefore run a single-agent enumerator and a pooled multi-sample enumerator at matched token budgets. We also have annotators judge, on a 300-claim sample, whether each generated context proposition names a genuinely omitted, relevant fact. If the single-agent enumerator matches debate, the contribution narrows to rhetoric-blind proposition scoring, and debate should be dropped for cost.
Why not just let the debate judge emit a calibrated score? Asking the D2D-style judge for its probability over verdict labels yields continuity but keeps the judge reading persuasive text. MADR's 4.99-versus-30% gap and the fluency bias documented in The Silent Judge predict that this judge will track rhetoric. Our perturbation test below measures that directly.
Why not just fine-tune a verifier or a distributional judge? X-Fact's out-of-domain collapse to about 15–16% is the cautionary result for fine-tuned classifiers. DiVA's gains over FactScore came with search, not from the regression head alone. A distributional judge trained on graded labels is still a legitimate competitor for the final mapping, so we include it as the aggregator variant fit on proposition features. It is not a replacement for the propositions.
Proposed Experiments and Decisive Checks
The main evaluation reports macro F1 on all three X-Fact splits, with per-class F1 broken out for the three intermediate classes. On VeriTaS we report MSE on the standard and longitudinal splits, plus Weibo21 F1 as a sanity check against Debate-to-Detect. As a transfer probe, the literal-versus-implied construction is also run on AggreFact XSum, where MADISSE reports 75.1% balanced accuracy.
Three checks decide the proposal. First, persuasion invariance: we rewrite the debaters' arguments for wrong propositions in more fluent, confident prose and measure how often each judging method flips. Rewriting changes only the arguments, not the propositions, so our scorer should flip far less than the transcript judge; if it does not, proposition wording is leaking rhetoric. Second, the gating test: we apply LASE-style gating, skipping debate when literal propositions score with high confidence, and measure the accuracy loss on Partly-True claims specifically. We predict that loss is large, which would show why misleading claims need debate even when literal confidence is high. Third, the leakage test: we compare the rise in error from pre-cutoff to post-cutoff VeriTaS claims across methods. Error should rise less for a method whose verdict depends on retrieved evidence than for one that can recall memorized fact-checks.
Calibration of the final graded scores is reported with expected calibration error. Cost is reported in tokens per claim, so that any accuracy gain is weighed against what it consumed.
References
- Debate-to-Detect: Reformulating Misinformation Detection as a Real-World Debate with Large Language Models
- Faithful, Unfaithful or Ambiguous? Multi-Agent Debate with Initial Stance for Summary Evaluation
- Can LLMs Produce Faithful Explanations For Fact-checking? Towards Faithful Explainable Fact-Checking via Multi-Agent Debate
- MAD-Sherlock: Multi-Agent Debate for Visual Misinformation Detection
- Evaluating Text-to-Visual Generation with Image-to-Text Generation
- TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answering
- Efficient Multi-Agent Reasoning via Confidence-Guided Adaptive Debate
- Enhancing Multi-Agent Debate System Performance via Confidence Expression
- DiVA: Fine-grained Factuality Verification with Agentic-Discriminative Verifier
- Distributional LLM-as-a-Judge
- VeriTaS: The First Dynamic Benchmark for Multimodal Automated Fact-Checking
- X-Fact: A New Benchmark Dataset for Multilingual Fact Checking
- The Silent Judge: Unacknowledged Shortcut Bias in LLM-as-a-Judge