Claim-Level Committee Agreement as a Training Mask for Factual Generation

Language models write fluent paragraphs in which true and false statements sit side by side. FActScore showed that factual precision falls on rare entities and as outputs grow longer (FActScore). Most remedies for these hallucinations (fluent but false or unsupported statements) work only at inference time: they score a finished answer and flag suspect sentences. Semi-supervised learning has an older and different answer to the problem of trusting a model's own outputs. In consensus pseudo-labeling, a model's predictions on unlabeled data are kept as training labels only when several independent models agree on them, because separate models rarely make the same mistake. We propose applying that idea at the level of individual facts and using the result to retrain the generator. A model writes long-form text. The text is split into atomic claims, each a single checkable statement. A committee of models from different families checks each claim. Fine-tuning then passes gradient only through the tokens of claims the committee confirms, and the tokens of disputed claims are masked out.

The proposal tests whether a committee's agreement on individual facts, with no retrieval and no human labels, can serve as a training signal that lowers the generator's own hallucination rate. It also tests whether this works without the generator learning to say less. The question matters because the two existing families of methods each cover half of it. Consensus-based hallucination detectors do not train the generator. Generators trained on pseudo-labels take whole accepted outputs, errors included.

Core Claim

The claim is that masked fine-tuning on claim-level cross-model agreement raises the factual precision of long-form generation (FActScore) above two simpler options: fine-tuning on unfiltered self-generations, and fine-tuning on outputs filtered at the sentence or document level. It also beats single-model sampling consensus when inference compute is matched. And it does this while keeping the number of supported facts per answer roughly constant.

The closest prior work falls into three groups. The first is consensus pseudo-labeling in language. Weak-to-Strong Reasoning fine-tunes a strong model on math problems where it and a weak model reach the same final answer. Dual-Teacher Agreement keeps synthetic translation pairs only when two teachers agree at the sentence level. DC-W2S filters step-level reward labels using agreement across judges and domain-embedding neighbors. All three accept or reject whole units: a final answer, a sentence, or a step. None can keep the true half of a paragraph and drop the false half. That ability is what long-form factuality needs, since FActScore shows errors are mixed within a single text.

The second group is consensus-based detection. SelfCheckGPT checks sentences against the same model's stochastic resamples. ARS trains a probe on the model's internal states, labeled by whether perturbed decodings agree with the original answer. OPDAI pseudo-labels SHROOM data by multi-LLM agreement. All three produce scores or discriminators, and none feeds the agreed content back into the generator.

The third group supplies the training mechanism we reuse. VeriFY masks the loss on hallucinated stages of self-verification traces, but it needs ground truth to decide which spans to mask. We replace that ground truth with committee agreement. MedFact-R1 does pseudo-label fine-tuning for medical factuality, but its labels come from a single unfiltered generator. It therefore motivates the filtering step rather than competing with it.

Research Program

The pipeline starts with the student, an open roughly 8B model, writing biographies for a large set of entity names. Each biography is split into atomic claims using the FActScore decomposition procedure. Each claim is then turned into a short probe question whose answer is the claim's key slot, such as a date, a place, or a role.

Committee members answer the probe, not the full prompt. This design choice addresses the paraphrase problem raised by Dual-Teacher Agreement. Agreement is checked on the fact itself, by entailment between a member's answer and the claim, so wording and style are free to vary. The committee contains two or three models from families other than the student's, for example Mistral-7B, Gemma and Qwen. We use different families because Asymmetric Tri-training shows that agreement filters work only when the voters are actually diverse. Each member answers the probe $k$ times. A claim is confirmed only if every member's majority answer entails it.

Confirmed claims are aligned back to token spans in the original biography, and the student is fine-tuned with a masked loss:

$$ \mathcal{L} = -\sum_{t} m_t \log p_\theta(y_t \mid y_{<t}, x), $$

Here $x$ is the prompt and $y_t$ is the $t$-th token of the student's own biography. The mask $m_t$ is 1 if token $t$ lies in a confirmed claim or in connective text that asserts no fact, and 0 if it lies in a disputed claim. The model is therefore never trained to reproduce a disputed statement. It is also never pushed toward silence, which would happen if whole outputs were dropped or abstentions were inserted. We run two or three rounds of generating, filtering and fine-tuning, and track whether the committee's confirmation rate and the student's precision rise together.

The program also includes a planned analysis of where the method should fail. TruthfulQA shows that models trained on overlapping web text repeat the same human misconceptions, and that larger models do so more. On such items a committee will agree confidently and wrongly. We measure this directly rather than assume it away.

Baselines

The naive reference point is the untouched student, along with the student fine-tuned on all of its own biographies with no filtering. Dual-Teacher Agreement found that unfiltered synthetic data fell below the gold-only baseline, at 41.8 versus 42.7 chrF++. That result suggests unfiltered self-training may actively hurt, and our experiments will check whether it does here.

The strongest existing black-box competitor is single-model sampling consensus, implemented as SelfCheckGPT. SelfCheckGPT reports AUC-PR 93.42 for detecting non-factual sentences on WikiBio GPT-3. We run it as a training filter. The student resamples its own biography, SelfCheckGPT scores each sentence, and the same masked loss is applied at sentence granularity. The number of model calls spent on SelfCheckGPT is set equal to the number spent on the committee, so the comparison isolates where the agreement comes from rather than how much compute was used. We found no reported result on Lacuna for any consensus-filtered generator retraining evaluated by FActScore. The training-time comparison therefore has to be produced here.

The trained-verification competitor is VeriFY, which reports hallucination-rate reductions of 9.7 to 53.3 percent with recall drops of 0.4 to 5.7 percent on short-form QA. We reproduce it on the same student using its TriviaQA recipe and compare on the short-form transfer evaluations it uses, TriviaQA, NQ-Open and HotpotQA, with equal fine-tuning budgets.

Two ablations isolate the key ingredient. The first is a granularity ablation: the same committee filter applied by dropping whole sentences or whole biographies that contain any disputed claim, rather than masking claims. The second is a source ablation: the claim-level filter with a committee made of the student's own resamples, so that agreement is intra-model rather than cross-model.

Finally, a privileged ceiling uses Wikipedia retrieval verification, the FActScore checker, as the filter. That checker reports less than 2% error against experts. To count as a success, the proposal must beat every non-privileged baseline on FActScore precision at matched or higher supported-fact counts, and must close a substantial share of the gap to the retrieval ceiling.

Why not just…?

Why not just filter with retrieval against Wikipedia? When a clean reference corpus exists, retrieval is the better filter, and we treat it as the ceiling rather than pretend otherwise. The case for committees rests on domains where retrieval coverage is thin. FActScore's own rare-entity breakdown points to these domains, as do private or specialist corpora. We test this case by reporting results by entity-frequency bucket. If consensus trails retrieval uniformly, including on rare entities, the method's value shrinks to settings with no corpus at all, and we will report that.

Why not just use the student's own sampling consistency? This is the source ablation above, and it is a real threat, because it is cheaper and needs only one model. The argument against it is that self-sampling cannot reject errors the model reproduces consistently. SelfCheckGPT acknowledges this limitation, and HaluEval finds models near chance when judging plausible errors in their own family's output. The compute-matched comparison settles the question. If intra-model agreement matches cross-model agreement, the committee is unnecessary.

Why not just drop any output that contains a disagreement? Whole-output filtering is what Dual-Teacher Agreement and Weak-to-Strong Reasoning do. Because most long biographies contain at least one error (about 73% of WikiBio GPT-3 sentences are inaccurate), dropping them would discard nearly all the data. It would also bias training toward short outputs about well-known entities. The granularity ablation measures both data retention and the resulting precision.

Why not just train the model to abstain when unsure? VeriFY reports that direct knowledge-probing abstention causes sharp drops in recall. Our masking never rewards omission, so we predict a smaller loss in supported facts per answer. This prediction is checked by reporting supported-fact counts alongside precision.

Proposed Experiments

The main experiment fine-tunes the student on biographies for entities disjoint from the FActScore evaluation set and evaluates FActScore precision and supported-fact count by entity frequency. Transfer is measured on TriviaQA, NQ-Open and HotpotQA hallucination rates using the VeriFY protocol, and MMLU is used to check that general knowledge is not damaged.

Before any training, we validate the filter itself. The committee's claim-level decisions are scored against the human sentence labels of the WikiBio GPT-3 set, mapping claims to sentences, and against the FActScore checker's labels. This tells us whether the pseudo-labels are precise enough to be worth training on.

The decisive checks come from the comparisons already described. Masking must beat dropping. Cross-model agreement must beat intra-model agreement at equal compute. Precision must rise without a fall in supported facts.

A final collusion test on TruthfulQA measures the share of committee-confirmed answers that the GPT-judge marks false, and whether training raises or lowers the student's truthfulness. A rise in confidently shared falsehoods would mark the boundary of the method. That boundary would show that agreement among models trained on the same web text cannot correct misconceptions the web itself contains.