Many systems try to make language models more factual with a loop: the model writes a draft, a critic (often another language model) points out problems, and the model rewrites. These loops reduce some factual errors, but they are slow, and they make the text longer at every pass. There are good reasons to suspect the extra length is not harmless. Models that are unsure of an answer tend to "babble," producing long, low-information text that is less accurate than their concise answers. The language-model judges that often act as critics also systematically prefer longer answers. Put these together and a refinement loop may do two things at once. It removes the specific error the critic flagged, and it adds new, unverified material that a length-favoring critic then approves. The result costs extra decoding time and may end up no more accurate overall.
This note proposes to measure that effect directly and to test a simple fix. The measurement splits each revision into the claims it deleted, the claims it replaced, and the claims it added, then fact-checks each group separately. The fix is subtractive refinement: the reviser may only delete flagged claims or replace them with text supported by retrieved evidence, and a revision is never allowed to be longer than the draft it revises. The central prediction is that added claims are the least accurate part of a revised answer. If so, a loop that cannot add should match or beat full rewriting on factual precision while decoding far fewer tokens and stopping sooner. That is where the latency saving comes from.
Core Claim
The hypothesis has three parts. First, in generate-critique-revise loops for open-ended factual answers, newly added sentences have lower factual precision than the sentences the model keeps from its draft. Second, the critic's preference for length is part of why such additions survive, and it can be exposed by comparing the critic's decision on a padded revision against a length-matched version with the same corrections. Third, a revision rule that permits only deletion and evidence-grounded replacement, under a hard no-growth constraint, recovers most or all of refinement's precision gain at a fraction of its decoding cost. The third claim has an obvious risk: the system can buy precision by saying less. The evaluation is therefore built around the number of supported claims kept, not precision alone.
Why this is open
Three findings motivate the work. Verbosity ≠ Veracity names verbosity compensation, meaning uncertain models pad their answers instead of answering briefly. It finds verbose answers on Qasper score 27.61% worse than concise ones, and it detects an uncertainty spike near the third generated token. That study covers short-form question answering and simply routes failures to a stronger model; it never looks inside a refinement loop. FactBench finds that Llama 3.1-405B is no more factually precise than the 70B model, because the larger model adds subjective modifiers ("well-known," "solid foundation") that its VERIFY checker cannot ground. FactBench reports this only as an observation. On the judge side, LLMEval reports that GPT-4 picks the longer answer 78.8% of the time when the two answers differ by more than 300 characters; humans do so 51.4% of the time. Quantifying Biases in LLM-as-a-Judge Evaluations estimates a length coefficient of 1.09 for GPT-4 versus 0.25 for humans on MT-Bench. Can External Validation Tools Improve Annotation Quality shows that judges prefer long, well-formatted answers that contain planted factual errors. None of these papers asks what happens when the length-biased judge is the critic inside a loop that edits a model's own text.
On the refinement side, VeReaFine is the closest existing method, and it supplies much of our pipeline. It uses a verifier that checks sentences and asks a 7B generator to rewrite or cut unsupported assertions, and it reports large recall gains on ClinIQLink. VeReaFine does not measure length, does not measure the accuracy of what rewrites add, and lets the generator rewrite freely; its authors also name verification latency as unsolved. Self-Refinement Tuning raises Tulu2-70B's AlpacaEval 2.0 win rate from 9.6% to 25.8%. By its own account, though, the responses get longer, and AlpacaEval rewards length, so the size of the real quality gain is unknown. That paper is the clearest case of the confound we want to separate out. Extrapolative Controlled Sequence Generation documents semantic drift over repeated edits and the lack of a good stopping rule; this constrains our design toward loops that end on their own. Latency-free alternatives such as Meta-Reflection and InTRO need white-box access to hidden states or a gold answer to condition on. Neither is available for open-ended factual text, so we do not compete with them. We work at the level of text edits, which also runs on API-only models.
Research Program
The first component is an edit-level audit of existing refinement loops. For each prompt we record every turn of the loop. Each draft is split into atomic claims, meaning short self-contained factual statements, using the decomposition and web-evidence checker from FactBench's VERIFY pipeline. VERIFY labels each claim as supported, unsupported, or undecidable. We then align claims between consecutive turns and label each claim as kept, deleted, replaced, or added. The quantity we care about is the precision of each group at turn $t$,
$$ P_g(t) = \frac{S_g(t)}{S_g(t) + U_g(t)}, $$
where $g$ is one of kept, replaced, or added, $S_g(t)$ is the number of supported claims in that group at turn $t$, and $U_g(t)$ is the number of unsupported or undecidable ones. The first hypothesis predicts $P_{\text{added}}(t) < P_{\text{kept}}(t)$ at every turn. We also track total length and the total count of supported claims across turns. This gives the "density versus veracity" trajectory that the research file notes nobody has recorded. To check whether the effect comes from uncertainty, we relate added-claim precision to the draft's early-token perplexity, following the third-token finding of Verbosity ≠ Veracity.
The second component is a critic audit. For each revision the critic accepted, we build a counterfactual that makes the same deletions and replacements but none of the additions, so it is equal in length or shorter. We then ask the same critic to choose between the two. This uses the rewrite-based treatment logic of the RATE protocol from Flattery, Fluff, and Fog, applied here to self-generated revisions rather than stylistic rewrites. If the critic prefers the padded version even when its added claims have lower VERIFY precision, then the length bias is doing causal work inside the loop.
The third component is the method. The generator is Qwen2.5-7B-Instruct, as in VeReaFine, with Llama 3.1-70B as a second scale. An in-loop checker flags unsupported sentences. This checker is the ANAH-v2 7B annotator (ANAH-v2), chosen because it judges single sentences, needs no pairwise comparison, and runs quickly. For each flagged span, the reviser may either delete it or replace it with a span whose content is entailed by a retrieved passage; the replacement is checked with the same annotator before it is accepted. Nothing may be appended, and the revision's token count cannot exceed the draft's. Because only flagged spans are regenerated, each turn decodes tokens for the replacements alone, not a full rewrite. The loop stops when a turn flags nothing, which removes the need for a separate stopping rule. As an optional gate, we skip refinement entirely when early-token uncertainty is low. We report the gate separately so it does not inflate the main result.
We deliberately keep the evaluator apart from the in-loop checker. VERIFY is used for scoring, and ANAH-v2 is used inside the loop. To make sure gains do not come from matching a particular checker's quirks, one run swaps the two roles.
Baselines
The naive reference point is a single pass with no refinement. We also run a variant of that single pass whose prompt asks the model to be concise, with no refinement, because this is the cheapest way to attack length. Both use the same generator, retrieval index, and decoding settings as every other condition.
The strongest existing competitor is a VeReaFine-style verifier loop. We reimplement its sentence check followed by rewriting, allowing up to three turns, and run it with the same 7B generator and the same retrieval. The verifier is ANAH-v2 here rather than their medical 8B model, so that only the revision rule differs between the two systems. On its home benchmark, VeReaFine reports a 75th-percentile (P75) recall of 0.559 on ClinIQLink short-inverse questions, against 0.358 for the base 7B model and 0.539 for a 32B model. Beating it on ClinIQLink means matching that P75 within noise while decoding substantially fewer tokens. On FactBench it means higher VERIFY precision with no loss in supported-claim count. For the open-domain setting, a second strong competitor is a Self-Refinement-Tuning-style prompted loop that asks the model for a critique and then a full rewrite. It uses a language-model judge as critic, as that paper does. It represents the "free rewrite with a biased critic" condition. No reported result was found on Lacuna for either loop on FactBench or HaluEval, so we will run both ourselves.
The decisive ablations isolate the proposal's two ingredients. One keeps the subtractive rule but removes the no-growth cap, so replacements may be longer. The other applies the cap to free rewriting, so text may be added only if other text is removed. A third is a delete-only filter: flag once with ANAH-v2 and drop every flagged sentence, with no replacement and no loop. All systems share the same prompts, retrieval, and a fixed wall-clock and token budget per prompt, and no system gets extra tuning passes on test data.
Why not just…?
Why not just prompt the reviser to stay concise or add nothing? Verbosity ≠ Veracity already found that explicit "answer in under three words" instructions fail most when the model is uncertain. Verbosity compensation stayed near 74% for Mistral-7B and above 50% for GPT-4 on some tasks despite the instruction. Uncertainty is also exactly the condition under which a model is revising a flagged claim. Still, a revision prompt is a different setting from a first draft, so we include the concise-prompt loop as a baseline. If it matches subtractive refinement on added-claim precision, the hard constraint is unnecessary, and we will report that.
Why not just length-correct the critic or add a length penalty? Counterfactual data augmentation in Flattery, Fluff, and Fog reduced reward-model miscalibration only from 39.4% to 32.5%, so debiasing the judge leaves much of the bias in place. A length penalty also leaves padding and correct elaboration tied together: it punishes a longer answer whether or not the added claims are true. Our critic audit tells us how much of the problem sits in the critic and how much in the generator. If a length-matched critic alone closes the gap, the critic is the right place to intervene, and our method loses its advantage.
Why not just delete every flagged sentence once and skip the loop? This is the real threat. It is cheap, it cannot add padding, and it may capture most of the precision gain. Its likely weakness is that it lowers the count of supported claims and can empty an answer; FactBench's report that Gemini 1.5 Pro over-refuses 25% of hard prompts shows that erring toward saying less is a real failure mode. The delete-only ablation settles the question. If it matches subtractive refinement on both precision and supported-claim count, the conclusion is that refinement's factual value comes almost entirely from deletion. That would be a useful negative result, and we would recommend delete-only as the default.
Why not just use a bigger model? FactBench's result that the 405B model is no more precise than the 70B model, and VeReaFine's 7B-plus-verifier beating a 32B model, both suggest scale does not remove padding and may add to it. We include Llama 3.1-70B as a second generator so this can be checked directly.
Proposed Experiments
The main open-domain testbed is the Hard tier of FactBench, scored with VERIFY. VERIFY correlates with human judgment at 0.97 on factual units but only 0.73 on non-factual ones. We therefore hand-check a stratified sample of added claims, because the key comparison depends on those labels. Short-form tests use HaluEval QA and Qasper, where the Verbosity ≠ Veracity accuracy gap gives an external reference point, and high-stakes transfer uses ClinIQLink. The critic audit also runs on LongFact pairwise, where tool-augmented GPT-4o judging reaches over 80% accuracy against 63% without tools; that gives an upper reference for how accurate a critic can be.
We report VERIFY precision, the count of supported claims, answer length, decoded tokens per prompt, number of turns, and wall-clock latency. The audit supports the core claim if, on at least two generators, added claims show clearly lower precision than kept claims, and the critic prefers padded revisions over their length-matched counterfactuals at rates well above what VERIFY precision would justify. The method succeeds if subtractive refinement matches or exceeds the VeReaFine-style loop in precision, keeps its supported-claim count within a few percent, and decodes markedly fewer tokens. Refinement loops have been reported to cost two to three times single-pass latency (Volcano), so we expect the saving to land most of that overhead back near single-pass cost. The idea fails outright if added claims turn out to be as accurate as kept ones. In that case padding is not the problem, and the latency cost of full rewriting is being paid for real gains.
References
- Verbosity $\neq$ Veracity: Demystify Verbosity Compensation Behavior of Large Language Models
- FactBench: A Dynamic Benchmark for In-the-Wild Language Model Factuality Evaluation
- LLMEval: A Preliminary Study on How to Evaluate Large Language Models
- Quantifying Biases in LLM-as-a-Judge Evaluations
- Can External Validation Tools Improve Annotation Quality for LLM-as-a-Judge?
- VeReaFine: Iterative Verification Reasoning Refinement RAG for Hallucination-Resistant on Open-Ended Clinical QA
- Teaching Language Models to Self-Improve by Learning from Language Feedback
- Extrapolative Controlled Sequence Generation via Iterative Refinement
- Meta-Reflection: A Feedback-Free Reflection Learning Framework
- In-Token Rationality Optimization: Towards Accurate and Concise LLM Reasoning via Self-Feedback
- Flattery, Fluff, and Fog: Diagnosing and Mitigating Idiosyncratic Biases in Preference Models
- ANAH-v2: Scaling Analytical Hallucination Annotation of Large Language Models
- Volcano: Mitigating Multimodal Hallucination through Self-Feedback Guided Revision