Belief Tags Before Claims: Training Models to Announce Confidence Before They Commit to a Fact

Language models trained to be more factual tend to become terse and to abstain often. The strongest current recipe, from The Curious Case of Factuality Finetuning, uses a probe to read the model's internal states. The model generates text, the text is split into atomic claims, and a small classifier trained on hidden activations (a "probe") judges whether the model internally believes each claim. Claims the model does not believe are deleted, and the model is fine-tuned on what remains. This raises factuality sharply, but the threshold is fixed at data-construction time. The resulting model gives shorter, less detailed answers, and users cannot choose how cautious it should be.

We propose a different use of the same probe. The probe's verdicts are kept as data rather than used to delete claims. Each claim in the model's own generations is preceded by a discrete "belief tag", a single token giving the binned probe confidence. The model is fine-tuned to predict that tag before writing the claim. At inference, a user-set threshold acts on the tag. If the model predicts a low-belief tag, decoding skips that claim and moves to another one before any of its tokens enter the context. The two-sentence version: we teach the model to say how much it believes a fact before it states it, using labels that come from its own internals, and then use that one token as a cheap, adjustable gate. Because the gate fires before the claim starts, it acts at the one point where hallucination is still detectable. It also lets the model replace a doubtful fact with a believed one, so the answer does not simply get shorter.

Core Claim

A model fine-tuned on its own generations, with every atomic claim prefixed by a probe-derived belief tag, can be steered at inference along the factuality–informativeness curve with one parameter. We claim three things. First, at matched factual precision, this model states more supported facts per answer than Newman et al.'s fixed filter. Second, it states more supported facts than the obvious inference-time alternative of generating freely and then deleting low-belief claims. Third, the tags work because they come from the model's internal belief rather than from external truth labels. If tags built from verified truth labels do as well, the interpretability ingredient is unnecessary and the claim fails.

Why This Is a Different Idea

Three lines of prior work come close, and each misses one piece.

The first line uses control tokens for continuous attributes. Controlling Grammatical Error Correction Using Word Edit Rate bins edit rate into five prefix tokens, and MUSS conditions on length and lexical-complexity tokens. These supply our method, but they control only surface statistics. They are also loose: Hotate et al. report that the output lands in the requested bin only 53–62% of the time. We place the tag inline before each claim, rather than once as a prefix, and enforce the threshold by masking during decoding instead of hoping the model complies.

The second line attaches factual control codes to synthetic data. Counterfactual Data Augmentation improves Factuality of Abstractive Summarization builds those codes from rule-based entity swaps, and it gains 3.2 NLI points on CNN/DailyMail at some cost in ROUGE. Controllable Factuality in Document-Grounded Dialog Systems Using a Noisy Channel Model offers a factuality dial, but it needs a reverse model during beam search. Our dial costs one extra predicted token per claim, and its labels reflect what the model knows rather than which words were corrupted.

The third line reads hallucination signals from inside the model but never feeds them back into generation. Bold Claims or Self-Doubt? classifies belief states from hidden activations, and SHINE detects fabrication by perturbing entity embeddings. Both are diagnostic only. Token-Guard does intervene during decoding. It prunes tokens based on hidden states and backtracks over segments, raising average F1 on Llama-3.1-8B-Instruct from 28.3 to 51.0, but it changes no weights and needs multi-stage thresholds. Our approach moves the readout into training data, so at test time the model performs the gating itself.

The timing of the tag is motivated by First Hallucination Tokens Are Different from Conditional Ones. That paper finds that the first token of an unsupported span can be detected at AUROC around 0.8, but later tokens fall to 0.5–0.6, because the model treats its own error as context. A tag placed immediately before a claim targets exactly that first-token window.

Research Program

Data construction. We sample long-form generations from Mistral-7B on the Bios, Plots, and Medical prompts used by Newman et al. Each generation is split into atomic claims and scored with a logistic-regression belief probe, following their recipe. Probe scores are binned into five equal-frequency bins, as Hotate et al. do for edit rate. The training sequence becomes the original text with a bin token inserted before each claim. Nothing is deleted, so low-belief claims stay in the data, labelled as low-belief.

Training and decoding. Standard fine-tuning teaches the model to predict both tags and claims. At inference, the decoder works with a threshold τ. Whenever the model emits a tag below τ, that claim is masked out and the model is resampled from the tag position, conditioned to produce a different claim or a sentence boundary. Sweeping τ traces a curve from cautious answers to detailed ones.

Interpretability analysis. The goal is to test whether the model has learned to report its own internal state or only a surface habit. We take hidden states at the tag position on held-out prompts and measure how well the model's predicted tag agrees with the original probe applied to the claim it then writes. As a causal test, we add or subtract the probe direction at the tag position, in the style of Activation Addition, and check whether predicted tags move as expected. If tags change under this steering, the model is reading its belief representation. If they do not, the tags are probably keyed to surface cues such as how rare an entity name looks.

The familiarity tension. Newman et al. find that training on unfamiliar gold facts teaches fabrication. PREREQ-TUNE finds that fictitious data improves grounding. Our design gives a direct test between them. We train one variant whose tags come from the internal probe and another whose tags come from external verification (FActScore support labels). Newman's view predicts the probe-tag variant wins on rare entities, because truth labels on facts the model never knew teach it to state unknown facts confidently.

Baselines

Base model and naive references. The do-nothing reference is the unfinetuned Mistral-7B. On Bios, Newman et al. report 29.6% factuality for this model, and 54.9% after fine-tuning on gold data. A second naive reference is prompting the model to "only state facts you are sure of". Bold Claims or Self-Doubt? found that direct instruction and few-shot prompting both reached only a 0.661 truthful rate on Gemma-2-9b-it, against 0.992 for its hidden-state classifier. That result covers belief-type detection, not generation, so we rerun prompting as a generation baseline.

Strongest existing method. The main competitor is Epistemic Training from Newman et al., which reports 76.6% factuality on Bios with Mistral-7B. We reproduce it using the same generations, the same probe, and the same fine-tuning budget, and sweep its filtering threshold to give it a curve rather than a single point. Beating it means one of two outcomes. Either the belief-tag model matches its factual precision while stating more supported atomic facts per answer, or it reaches higher precision at an equal number of supported facts.

PREREQ-TUNE and Token-Guard. PREREQ-TUNE is evaluated on the FActScore biography task, where it reports 45.30% against 32.70% for standard SFT. It is the strongest weight-based alternative that uses no probe. Token-Guard is the strongest alternative that changes only decoding. We run both on the same prompts. Token-Guard is also compared on wall-clock throughput; it reports 164.76 normalized tokens per second on HaluEval.

Ablations isolating the key ingredient. Four ablations each remove one part of the method. Truth-label tags replace probe scores with FActScore support labels, testing whether internal belief matters. Entropy tags bin the first-token entropy of each claim instead of the probe score, testing whether a cheap output statistic is enough. Random tags test whether the tag position simply acts as a pause. A single global prefix token (Hotate-style) tests whether the claim-level, inline placement matters. All variants use identical data, steps, and threshold sweeps.

Why not just…?

Why not just generate freely, then delete low-belief claims with the probe at inference? This is the most serious threat, because it also gives a dial and needs no training. Its weakness is that a deleted claim is gone. The answer gets shorter, the remaining text may lose coherence, and every response needs atomization and probing at inference. Our method can replace a doubtful claim with a believed one. The decisive measurement is the number of supported facts at matched precision. If post-hoc deletion matches us on that measure, the training step adds nothing, and we will report it as the better method.

Why not just use FActScore-verified labels as control tokens? This route skips interpretability entirely. The truth-label ablation tests it directly. Newman et al.'s finding that unfamiliar gold facts induce fabrication predicts it will fail on rare entities, which FActScore shows are the main source of errors. If truth labels match probe labels across all rarity levels, the internal-belief claim fails.

Why not just threshold first-token entropy, with no fine-tuning? Snel and Oh show first-token entropy already reaches AUROC around 0.8. We run entropy gating as both an inference-only baseline and a training-tag ablation. The probe should add the most value on confident errors: Bold Claims or Self-Doubt? shows that some hallucinations are repeated consistently, and such errors may be low-entropy. We report results separately on that subset.

Why not just steer activations along the probe's truth direction? MuseCoco reports that internal-layer conditioning disrupts pretrained representations compared with prefix tokens. Zhang et al. show that intervening in certain critical layers collapses the model (cornerstone layers). Neither result concerns factuality, so we run contrastive activation addition (Steering Llama 2 via Contrastive Activation Addition) with a sweep over strength. We then compare its precision–detail curve and its fluency, measured by perplexity under the base model.

Proposed Experiments

The main experiment uses Mistral-7B, plus Llama-2-7B as a replication, on Bios, Plots, and Medical, scored with FActScore-style atomic verification. For every method we report three numbers at each threshold: factual precision, number of supported facts per response, and abstention rate. Methods are summarized by the area under the precision-versus-supported-facts curve. We also report controlling accuracy, meaning how often the realized claim-level belief matches the requested threshold, which can be compared directly with Hotate et al.'s 53–62%. Results are split by entity frequency to test behaviour on the rare tail. PopQA and HotpotQA test whether the method transfers to short-form questions. For HotpotQA, PREREQ-TUNE's bridging accuracy of 26.22% is the reference; no reported result was found on Lacuna for Epistemic Training on these sets.

Three checks decide the outcome. The proposal succeeds only if the belief-tag model's curve lies above both Epistemic Training's swept curve and post-hoc probe deletion on supported facts at matched precision. It must also beat truth-label tags on rare entities. Finally, predicted tags must shift when the probe direction is steered at the tag position. If the first check fails, a deletion pipeline is enough. If the second fails, external labels are enough. If the third fails, the model has learned a surface heuristic rather than a readout of its own belief, and we would report the method as a heuristic.