Language models often "show their work" before answering, writing out intermediate steps (chain-of-thought prompting). Auditors want those steps to be the real reason for the answer. Several studies report that bigger models are less faithful to their own reasoning: when a step is altered, a large model tends to ignore the change and give its original answer anyway. Yet one audit of eleven frontier models finds the opposite ordering. We think much of this disagreement comes from the instrument, not the models. Automated perturbations are often easy to spot as errors, and many test items can be answered by a large model with no reasoning at all. In both cases, ignoring the altered step is ambiguous: it may mean the model is unfaithful, or it may mean the model is rightly correcting an obvious error it never needed.
We propose to separate the two readings with a small amount of targeted human editing. Annotators make one minimal, natural-sounding edit to a single step of a model's reasoning, in the "surgical edit" style that has been shown to take about 21 seconds per item. Some edits are errors that change which answer the chain implies. Others are paired with a matching edit to the question, so the altered step is correct and the right answer really changes. We then measure how each model responds across a family of sizes, grouping items by whether the model could answer them without reasoning. If the apparent decline with scale mostly disappears on items that genuinely need the chain, the scaling story changes. The same human-verified edits then become training data for a specific, checkable behavior: either follow the edited chain or say explicitly that it is wrong, but never quietly override it.
Core Claim
The claim has two parts. First, measured faithfulness depends on three things beyond model size: how detectable the perturbation is, whether the edit decisively changes the implied answer, and whether the model can answer the item without the chain. Once all three are controlled, the inverse-scaling trend should shrink substantially. It should survive mainly on items the model can already answer in a single pass. Second, a behavior we call flag-or-follow can be trained from a few thousand human-verified single-step edits, and it transfers to existing faithfulness benchmarks better than automated distractor or rewriting pipelines at matched cost. Flag-or-follow means the model either adopts the consequence of an edited step or explicitly flags that step as wrong. The failure it rules out is silent correction.
The second claim depends on the first. If automated perturbations, once filtered for detectability, turn out to be as decisive as human edits, then humans are not needed and we will say so.
Motivating and Constraining Prior Work
Three papers motivate the decline-with-scale concern. Measuring Faithfulness in Chain-of-Thought Reasoning truncates, corrupts, paraphrases, and pads model reasoning, and finds that a 13B model often depends on its chain more than a 175B one. RFEval inserts flawed reasoning openings into twelve reasoning-trained models and finds 49.7% of outputs unfaithful, with gpt-oss-120B less faithful than gpt-oss-20B. It also finds no significant correlation between accuracy and faithfulness. Counterfactual Simulation Training finds that Qwen3-235B is no more faithful out of the box than Qwen3-4B.
One paper constrains the story. The corruption-probe audit injects arithmetic swaps, logic flips, fact swaps, and step deletions into models' own reasoning. It reports pooled faithfulness of 56.1%, frontier models at 67–69%, and the smallest models at 35–42%. That is the reverse ordering. Rather than dismiss it, we treat it as evidence that the ordering depends on the perturbation and the items. Our design tests that directly.
A mechanism for the decline is suggested by Making Reasoning Matter. It argues that models which already know the answer produce the chain as after-the-fact justification. Faithful and Unfaithful Error Recovery adds that GPT-4, Claude-3 Opus, and Llama-3 70B respond in opposite directions to larger injected errors. Both results point to perturbation properties, not size alone.
On the method side, we borrow from Human-in-the-Loop through Chain-of-Thought. Its Manual Correction System has humans make a single Modify, Add, or Delete edit to one step. It reports that up to half of reasoning errors trace to one faulty step, and that an edit takes 21s versus 49s for solving from scratch on AddSub. It used these edits only to raise accuracy of one GPT-3 model at inference time. It never asked whether models actually use the edited step, never varied scale, and never trained on the edits. Those three gaps are what we fill.
The instance-level labels in FaithCoT-Bench warn that fluent large-model explanations defeat automated faithfulness detectors. This is one reason we define our target behavior by observable outcomes (did the answer follow, was the step flagged) rather than by a judge's opinion of the prose.
Research Program
The edit benchmark. We draw items from GSM8K, SVAMP, StrategyQA, and CLUTRR. We also add a slice of recent competition problems, following the contamination-avoidance logic of BrokenMath. This slice lowers the chance that large models have memorized answers.
For each item we sample a reasoning chain from a reference model. An annotator then produces two edits to one step:
- Decisive-error edit. The annotator changes one step so that the chain, carried through faithfully, implies a specific different answer, which the annotator records. The step must read as natural.
- Consistent counterfactual edit. The annotator edits the same step and makes the matching minimal change to the question. The edited step is then correct for the new question, and the new correct answer is recorded.
A second annotator checks that each edit is decisive. A detectability screen follows: a classifier trained to separate edited steps from original steps is run on held-out edits, and edits it flags too easily are revised.
For every model and item we also record no-reasoning answerability: whether the model answers correctly when forced to answer immediately.
The scaling analysis. We evaluate the Qwen3 family from 4B to 235B and gpt-oss-20B/120B. These are open-weight models, which lets us force the edited chain into the model's context window. Each model receives the original question with the edited chain up to and including the edited step, then continues on its own. Outcomes fall into four classes, scored by answer extraction plus a flagging check that humans verify on a sample:
- followed: the model gives the edit-implied answer;
- flagged: the model explicitly identifies the step as wrong;
- silently corrected: the model gives the original answer with no acknowledgment;
- other.
For consistent counterfactual edits, only "followed" is correct. The primary quantity is the flag-or-follow rate, reported as a function of model size, separately for items answerable and not answerable without reasoning. We run the same analysis on matched automated perturbations of the same steps (FRIT-style distractors and corruption-probe corruptions). This separates the effect of edit quality from the effect of item answerability.
Training. Human-verified edits are turned into supervised targets. Error edits are paired with continuations that flag the step and recompute. Consistent counterfactual edits are paired with continuations that follow. Silent-correction continuations sampled from the model are used as rejected examples for preference training. We fine-tune Qwen3-8B and, with low-rank adapters, the largest size our compute allows. Evaluation happens on held-out edits and on external benchmarks.
Baselines
Naive reference points. The untouched model under the same forced-prefix protocol is the "do nothing" point. We also include the prompting intervention from BrokenMath, an instruction to "be critical." BrokenMath reports it reduced DeepSeek sycophancy by 34% without eliminating it. It costs nothing, so any training method must beat it.
FRIT. FRIT is the strongest automated step-level competitor. It replaces steps with style-matched distractors drawn from Wikidata and arithmetic facts, labels a step causally important if the swap changes the answer, and trains faithful-over-unfaithful preferences. It reports a 7.6 percentage-point GSM8K accuracy gain for Mistral-7B and a 6.0-point StrategyQA gain for Qwen3-8B. We run FRIT on the same source items and the same base models. Its annotation budget is converted to generation compute that yields the same number of preference pairs, and it gets the same hyperparameter search as our method. Beating FRIT means a higher flag-or-follow rate on held-out human edits and on RFEval, with equal or better task accuracy.
Counterfactual Simulation Training. CST has a language model rewrite chains until they admit the cue driving the answer, then trains on the rewrites. It reports a 35-point gain in monitor accuracy on cue-based counterfactuals but only 2 points on generic ones. That split makes it the right competitor for our generic, step-level edits. We apply its rewriting loop to our items with matched rewrite budget.
ProFIL. ProFIL filters reinforcement-learning rollouts with a probe that detects when the model has already committed internally. It reports near-100% reduction of post-commitment "theater" on GSM8K and +5.2pp accuracy. It targets a different failure, but we report it as a reference on GSM8K.
Ablations. The key ablation isolates the human ingredient. We use the identical training recipe on LLM-generated edits of the same steps, both with and without our detectability screen. A second ablation drops the consistent counterfactual half, to test whether "follow" examples are needed to prevent the model from learning to reject every edit.
For flag-or-follow rate on human-edited steps, no reported result found on Lacuna for any baseline. All numbers on the new benchmark will be produced here.
Why not just…?
Why not just generate the edits automatically, as FRIT and CST do? This is the main threat. The detectability-screened LLM-edit ablation is designed to settle it. The evidence for human edits is indirect. MCS shows single human edits are cheap and target the step that actually matters. CST's gains collapse from 35 points on stylized cues to 2 points on generic perturbations, which suggests automated generic edits are weak signals. If screened automated edits match human edits on both measurement and training, the human component should be dropped, and we will report that.
Why not just stratify existing benchmarks like RFEval by no-reasoning answerability? This needs no humans and will be our first experiment. It can reveal whether answerability drives the decline with scale. What it cannot do is produce consistent counterfactual items, where following the chain is unambiguously correct. Without those, silent correction cannot be distinguished from legitimate error repair, which is exactly the confound in the RFEval and corruption-probe disagreement.
Why not just use symbolic execution? Faithful CoT makes the answer faithful by construction, since a deterministic solver runs model-written code. It reaches 71.9% on CLUTRR versus 48.5% for standard reasoning. But translation errors remain unfaithful in a subtler way, and the method cannot handle StrategyQA-style open reasoning. We include it on CLUTRR and GSM8K to mark what "faithful by construction" achieves there.
Decisive Checks
The measurement claim fails if the decline in flag-or-follow rate from Qwen3-4B to 235B is as steep on items that cannot be answered without reasoning as on items that can.
The human-data claim fails if screened automated edits match human edits within noise, on both the scaling curves and the training transfer.
The training claim is supported only under three conditions. The fine-tuned model must raise flag-or-follow on held-out human edits. It must reduce unfaithful outputs on RFEval below the base model and below FRIT and CST at matched budget. And it must avoid learning to follow every edit. We check the last point by requiring the model not to follow the injected mistake on BrokenMath's false statements, and to keep GSM8K and StrategyQA accuracy within one point of the base model. A model that simply learns to always follow edits would score well on consistent counterfactuals but would fail on BrokenMath, which is why that check is included.
References
- Measuring Faithfulness in Chain-of-Thought Reasoning
- RFEval: Benchmarking Reasoning Faithfulness under Counterfactual Reasoning Intervention in Large Reasoning Models
- Counterfactual Simulation Training for Chain-of-Thought Faithfulness
- Auditing Chain-of-Thought Faithfulness for Trustworthy AI: A Reproducible Corruption-Probe Protocol Across Eleven Frontier LLMs
- Making Reasoning Matter: Measuring and Improving Faithfulness of Chain-of-Thought Reasoning
- Faithful and Unfaithful Error Recovery in Chain of Thought
- Human-in-the-Loop through Chain-of-Thought
- FaithCoT-Bench: Benchmarking Instance-Level Faithfulness of Chain-of-Thought Reasoning
- BrokenMath: A Benchmark for Sycophancy in Theorem Proving with LLMs
- FRIT: Using Causal Importance to Improve Chain-of-Thought Faithfulness
- Drop the Act: Probe-Filtered RL for Faithful Chain-of-Thought Reasoning
- Faithful Chain-of-Thought Reasoning