Teaching Small Fact-Checkers to Trust the Right Source

When a fact-checking system searches the web and finds documents that disagree, it has to decide whom to believe. Large language models do this noticeably better when the prompt tells them who each publisher is, such as its political lean and its factual track record. But this works only for large models, with long prompts, at high cost per claim. Small models that could run at scale either never see this information or cannot use it reliably. This note proposes to train a small model to do the adjudication itself. A large "teacher" model writes step-by-step explanations of which sources it trusts and why, and a small "student" learns from them. The central twist is counterfactual training: we deliberately swap the credibility profiles of the supporting and refuting outlets, so the student is rewarded only when its verdict actually follows the evidence about the sources.

The contribution is a test of whether the student really uses source credibility, and a training recipe that makes it do so. A student can reach good accuracy on a benchmark by leaning on whatever it memorized during pretraining, or on how many documents agree, while ignoring the credibility information entirely. We measure this directly. We flip the credibility labels and check whether the verdict flips with them, and we run the same flip on irrelevant metadata as a control. We then ask whether this sensitivity survives when the explanation is dropped at test time, or whether the student must still write a short trace.

Core Claim

We make two claims. First, a 3-billion-parameter student trained on credibility-reasoning explanations from a teacher, plus profile-swapped counterfactual examples, can approach the conflict-resolution accuracy of a large model prompted with publisher background. Second, only the counterfactually trained student will change its verdict when credibility is swapped. Students trained on labels alone, or on explanations without counterfactuals, will match accuracy on the natural data but fail this intervention test.

We also expect a dividing line on how much reasoning can be hidden inside the weights. Retrieved evidence and source profiles change with every claim, much like the novel search problems where fully internalized reasoning broke down. We therefore predict that a student emitting no trace keeps its accuracy but loses much of its sensitivity to credibility. A student emitting a very short scaffold, only a one-line ranking of which sources it trusts, should recover that sensitivity.

Where This Sits

Three lines of work motivate the proposal.

  • Source credibility helps, but only at generation time. Resolving Conflicting Evidence in Automated Fact-Checking introduced the CONFACT benchmark. It found that appending publisher background to the prompt ("source background augmentation") beats both filtering out low-credibility documents and reranking them. It also documented two failures: the "majority vote trap," where repeated low-credibility copies win, and accuracy that drops as prompts lengthen. That paper uses large models at runtime and does no distillation. We take its prompting as our teacher and its benchmark as our testbed.
  • Real-world verification is costly and prone to ungrounded verdicts. HerO shows that strong verification on AVeriTeC currently relies on a fine-tuned 70B model with full reasoning at inference. It also does not profile sources. AVeriTeC observes that models reach verdicts using internal knowledge rather than retrieved evidence. This is the kind of shortcut our intervention test is designed to expose.
  • Consensus-based reliability scoring. RA-RAG scores source reliability by cross-checking claims across sources and then takes a weighted vote. This is cheap and robust to random noise. However, it assumes that agreement tracks truth, which is exactly what fails when misinformation is syndicated across many outlets.

The method comes from work on distilling step-by-step reasoning into smaller models.

  • Distilling Step-by-Step trains a small model to predict both the label and the teacher's explanation. It shows large gains in data efficiency over label-only fine-tuning.
  • SCOTT adds counterfactual training so that the student's answer depends on its own explanation. It found that larger students are more prone to ignoring their explanations. SCOTT perturbs explanations on closed-book claims. We instead perturb the source evidence in open retrieval, which SCOTT never touches.
  • Dual-Head Reasoning Distillation keeps the reasoning head only during training, making inference 96–142× faster on SuperGLUE.
  • Masked Distillation reports that full internalization falls to 41.7% on Countdown but recovers to 86.2% with a short suffix scaffold.

These two results constrain our design: they motivate testing no-trace, short-scaffold, and full-trace students against each other, rather than assuming that any one of them works.

Mujahid et al. supply the other half of the input: how to build the source profiles themselves. They produce outlet-level profiles of factuality and bias from MBFC-style reasoning. Their pipeline still calls a large model at inference time, and it only profiles outlets; it does not adjudicate claims.

Research Program

Inputs. Each training instance contains a claim, its retrieved documents, and a compact profile for each publisher. The profile gives factuality (low, mixed or high) and bias, drawn from CONFACT's publisher ratings or MBFC. We encode profiles as short structured tags attached to each document rather than long dossiers, because long prompts are known to hurt.

Teacher explanations. The teacher is Llama-3.1-70B with source background augmentation. It writes an explanation with a fixed shape:

  1. which documents support and which refute the claim;
  2. which publishers it trusts, and why;
  3. whether agreement among sources reflects independent confirmation or repetition;
  4. the verdict.

We keep only explanations whose verdict matches the gold label.

Counterfactual examples. For each conflicting claim, we exchange the credibility profiles between the supporting and refuting outlets. We then re-query the teacher, so that its explanation and verdict now favor the side that has become credible. These examples teach "follow the credible side," not "memorize the true answer." We also create "irrelevant swap" examples, which exchange metadata that should not matter (such as an outlet's founding year or country field) and keep the original verdict. These penalize a student that simply reacts to any metadata change.

To stop the student from memorizing outlet names, we train on half of the publishers and test on the held-out half. We also train a variant in which publisher names are replaced by anonymous IDs. Together these address the concern that credibility is dynamic rather than a fixed reputation per outlet.

Students. The students are Qwen2.5-3B and Llama-3.2-3B. Each is trained in three output modes:

  • Full trace: the student emits the full explanation, then the verdict.
  • Short scaffold: the student emits only the one-line source ranking, then the verdict (the analogue of Masked Distillation's suffix).
  • No trace: a verdict head whose explanation head exists only during training, following Dual-Head Reasoning Distillation.

The training loss sums a verdict loss and an explanation loss, following Distilling Step-by-Step, over both natural and counterfactual examples.

Baselines

Naive reference. The 3B student fine-tuned on claims, documents and gold verdicts, with no source profiles. We also report RA-RAG-style weighted voting on the same retrieved documents.

Label-only with profiles (the most dangerous competitor). The same 3B student sees the identical profile tags and is fine-tuned on verdicts only. It uses the same data, epochs, and tuning budget (the same number of learning-rate and weighting sweeps) as our method. If it matches our method on accuracy and on the intervention test, the explanations add nothing.

Upper reference: the prompted large model. The strongest existing method is Llama-3.1-70B with source background augmentation, as in Resolving Conflicting Evidence. That paper reports a 10% absolute F1 gain over the same model without background on CONFACT's ModC split. The file gives this only as a relative gain; no absolute CONFACT F1 appears on Lacuna, so we will rerun it ourselves.

On AVeriTeC, the strongest reported results are:

  • HerO: 75.2% veracity accuracy on the dev set, and an overall shared-task score of 0.57 (HerO).
  • GPT-3.5-turbo: 0.39 veracity F1 (AVeriTeC).

Ablations that isolate the key ingredient. - Explanation distillation without counterfactual examples. This is the obvious combination of Distilling Step-by-Step with source background augmentation. - Counterfactual examples with label-only training. This separates the contribution of the counterfactual data from the contribution of the explanations.

What counts as success. We count it as beating the baselines if both conditions hold:

  1. The student comes within a few F1 points of the prompted 70B model on the CONFACT human-verified split (HumC) at a fraction of its latency.
  2. The student shows a clearly higher credibility-flip rate than the label-only and no-counterfactual students, with no increase in flips under irrelevant swaps.

Why not just…?

Why not just filter out low-credibility sources before the model sees them? Resolving Conflicting Evidence tested both source filtering and credibility reweighting, and found that reasoning at generation time beat both. Song et al. and Glockner et al. show that misinformation often cites legitimate or predatory repositories that pass domain whitelists. We will still include filtering in front of the naive student as a baseline.

Why not just fine-tune the small model on labels with the profiles in its input? This is a real threat. Explanation distillation's data-efficiency advantage (Distilling Step-by-Step) may shrink when the input already carries the key signal. That is exactly why counterfactual label-only training is its own ablation. If that student matches ours on accuracy and flip rate, we will report that the counterfactual data, not the explanations, does the work. We will also report accuracy as a function of training-set size, where explanations are expected to matter most.

Why not just use consensus-weighted voting like RA-RAG? Voting is cheap. However, it treats repetition as corroboration, which Resolving Conflicting Evidence identifies as the majority vote trap. Our syndication stress test (below) directly measures whether it collapses as low-credibility copies are added.

Proposed Experiments and Decisive Checks

Benchmarks. We use CONFACT's ModC and HumC splits, with held-out publishers, and AVeriTeC with its temporal test split and its conflicting-evidence/cherry-picking label. AVeriTeC tests transfer to messier retrieval. There, source profiles come from the MBFC-style profiling of Mujahid et al. and are missing for some domains.

Metrics. We report:

  • macro-F1 and accuracy;
  • the credibility-flip rate, meaning the fraction of conflicting claims whose verdict follows the swapped profiles;
  • the irrelevant-flip rate;
  • expected calibration error;
  • tokens generated and throughput per claim.

Decisive check 1: the intervention test. On held-out publishers, a student that relies on credibility should reach a high credibility-flip rate and a near-zero irrelevant-flip rate. We predict that label-only students and students trained without counterfactuals will show low credibility-flip rates despite similar accuracy.

Decisive check 2: the syndication stress test. We inject 1, 3 and 5 near-duplicate low-credibility documents supporting a false claim. We then plot accuracy against the number of duplicates for our student, weighted voting, and the prompted 70B model.

Decisive check 3: how much reasoning can be hidden. We compare no-trace, short-scaffold and full-trace students on both accuracy and flip rate. If the no-trace student keeps its flip rate, the Masked Distillation failure does not carry over, and we get Dual-Head-level speed. If only the scaffolded student keeps it, adjudicating over per-query evidence needs a small amount of visible reasoning, and we will measure its cost in tokens.