Language models that "show their work" often do not use it. They settle on an answer through a shortcut, and the written reasoning is a justification produced alongside it. A small number of training methods now teach models to make their answers depend on their reasoning. Their own authors report a problem: once the model is fine-tuned further, that dependence erodes. This proposal borrows elastic weight consolidation (EWC), a continual-learning method that stops a network from forgetting old skills by penalizing changes to the weights those skills rely on most. We apply it to the specific skill of letting the reasoning decide the answer.
The key change is what "rely on" means. Standard EWC scores each weight by how much it affects the likelihood of the correct answer. For a model that rationalizes after the fact, that score is dominated by the direct shortcut from question to answer, so standard EWC would mostly preserve the shortcut. We instead score weights by how much they affect the gap between the answer the model gives under its own reasoning and the answer it gives under a counterfactually edited reasoning trace. We then consolidate those weights during later fine-tuning. The two-sentence version: find the weights that make the answer listen to the chain of thought, and make them stiff when the model is fine-tuned on something else.
Core Claim
Faithfulness gained through intervention training can be kept through later fine-tuning at a much lower accuracy cost when the consolidation penalty is weighted by causally mediated importance. Weighting by the usual answer-likelihood importance does not achieve this.
The two lines of prior work divide cleanly. Overcoming catastrophic forgetting in neural networks supplies the method. It introduced EWC, a quadratic penalty anchored at old weights and scaled by the diagonal Fisher information, where the Fisher score is the average squared gradient of the log-likelihood with respect to each weight. Elastic weight consolidation for better bias inoculation shows the method transfers to NLP debiasing: on FEVER, fine-tuning with EWC reached 82% on the original data against 79% for plain fine-tuning. That work concerns classification decisions only, with no generated reasoning.
On the faithfulness side, FRIT swaps reasoning steps for style-matched distractor facts, labels the steps that flip the answer as causally important, and trains with preference optimization. It documents "faithfulness drift" under continued optimization but offers no parameter-level remedy; this is the gap that motivates us. How Likely Do LLMs with CoT Mimic Human Reasoning? finds that ordinary supervised fine-tuning and preference tuning weaken the reasoning-to-answer link. This is why we expect downstream fine-tuning to undo faithfulness training. Measuring Faithfulness of Chains of Thought by Unlearning Reasoning Steps is the closest existing use of parameter-level intervention for faithfulness. It unlearns the knowledge behind a reasoning step from feed-forward matrices to test whether the answer depended on it, but only as an offline measurement for single examples. We turn a parameter-level sensitivity into a training-time constraint.
Research Program
Let $z$ be a question, $x$ the model's reasoning trace, $y$ its answer, and $\tilde{x}$ a FRIT-style counterfactual trace in which one causally important step is replaced by a distractor that, under the faithful model, flips the answer. The model's reasoning dependence on this example is
$$D(\theta) = \log p_\theta(y \mid z, x) - \log p_\theta(y \mid z, \tilde{x}),$$
which is large when changing the reasoning changes the answer and near zero when the answer is fixed in advance. The mediated importance of weight $\theta_i$ is $M_i = \mathbb{E}[(\partial D / \partial \theta_i)^2]$. The expectation runs over the counterfactual pairs already produced during FRIT training, so no new data are needed. Downstream fine-tuning on a new task then minimizes
$$\mathcal{L}(\theta) = \mathcal{L}_{\text{new}}(\theta) + \frac{\lambda}{2} \sum_i M_i (\theta_i - \theta^*_i)^2,$$
where $\theta^*$ are the weights after faithfulness training and $\lambda$ sets the stiffness. The only difference from standard EWC is replacing the Fisher score $F_i$ with $M_i$.
We also test one variant motivated by the shortcut argument. It uses importance $\max(0, M_i - \alpha F_i)$, which releases weights that matter mainly for the direct answer path. The variant stays in the study only if it beats plain $M_i$.
The program has three parts. First, we reproduce FRIT on Mistral-7B-v0.1 and Qwen3-8B on GSM8K and StrategyQA, and measure how much faithfulness is lost under two downstream fine-tuning regimes. The first regime is supervised fine-tuning on fluent human-written rationales (AQuA), which Bao et al. predict should reward rationalization. The second is supervised fine-tuning on a separate multiple-choice domain (ARC-Challenge and OpenbookQA). Second, we compare consolidation schemes along full accuracy–faithfulness frontiers obtained by sweeping $\lambda$. Third, we ask where $M$ concentrates. Tutek et al. localize step knowledge in the second feed-forward projection. Thought Anchors finds late-layer "receiver" attention heads that attend to load-bearing sentences. Measuring whether high-$M$ weights overlap with either gives a cheap check that the importance score tracks a real mechanism. If they do overlap, a layer-restricted penalty would also cut the memory cost of storing importance scores, which the EWC literature flags as a problem at 7B scale and above.
Baselines
The naive reference is downstream fine-tuning of the FRIT model with no protection. It defines how much drift there is to recover. The file reports FRIT's gains at stage one: +7.6 points GSM8K accuracy for Mistral-7B and +6.0 points StrategyQA accuracy for Qwen3-8B, in FRIT. It gives no reported number on Lacuna for the size of faithfulness drift, so measuring it is our first result.
Uniform L2 toward the FRIT weights penalizes all weight changes equally. Elastic weight consolidation for better bias inoculation found EWC dominated this on Pareto frontiers for classification. It is included because it is the cheapest possible consolidation.
Standard EWC uses the diagonal Fisher of answer likelihood given the model's own chain of thought, computed on the same examples used for $M$. It is the key ablation: it isolates whether causal mediation, rather than consolidation as such, carries the effect. We also run shuffled-$M$ as a control. It uses the same importance values randomly permuted across weights, which separates "these specific weights" from "this overall amount of stiffness."
The strongest practical competitor is replay: mixing FRIT's preference pairs back into downstream fine-tuning at a tuned ratio. LoRA-only fine-tuning (small low-rank adapters) is included as a structural limit on drift.
All methods see the same downstream data and the same number of optimizer steps. Each gets the same hyperparameter budget of eight settings per method, covering $\lambda$, the replay ratio, and the adapter rank. Methods are compared by the area under the frontier of faithfulness retention against downstream accuracy. Mediated consolidation beats a baseline if its frontier dominates at matched downstream accuracy, with a confidence interval over three seeds that excludes zero. It must beat standard EWC on faithfulness retention and at least match replay without access to replayed preference data.
Why not just…?
Why not just use standard EWC? Our central argument is that answer-likelihood importance is dominated by the direct question-to-answer path, the path Bao et al. identify behind "explainer" behavior. If so, standard EWC should keep accuracy while letting reasoning dependence decay. This is an empirical claim, and the standard-EWC arm tests it directly. If standard EWC matches mediated consolidation, the causal weighting is unnecessary and we will report that.
Why not just replay the faithfulness data? This is the real threat. Replay directly re-optimizes the objective we want to keep, and it may simply win. Our case for consolidation is practical. It needs only a stored importance vector rather than the whole counterfactual pipeline and preference data at every downstream fine-tune. That suits the common situation where the party doing downstream fine-tuning is not the party that did faithfulness training. The replay arm settles the empirical question. If replay dominates at equal compute, the contribution narrows to the diagnostic finding of where faithfulness lives in the weights, plus the combination of replay and consolidation. We will test that combination.
Why not just re-run FRIT after each downstream fine-tune? Re-alignment is possible, but it repeats a combinatorial intervention search every time, and FRIT itself reports that preference updates drift. We will report re-alignment cost in GPU-hours as a reference point. We do not treat it as a competitor at equal compute.
Why not just fine-tune with small adapters? LoRA limits how far the weights move but does not choose which directions to protect. The adapter arm shows whether low rank alone prevents drift. If it does, consolidation is only worth using for full fine-tuning.
Proposed Experiments and Decisive Checks
Faithfulness is measured four ways. The first is the FRIT-style distractor flip rate on held-out GSM8K and StrategyQA. The second is early-answering area over the curve from On the Hardness of Faithful Chain-of-Thought Reasoning in Large Language Models, which tracks how the answer probability moves as reasoning steps are revealed one by one. The third is the corrupted-trace treatment effect from Bao et al. The fourth, on a StrategyQA subset, is the parametric unlearning test of Tutek et al. It is chosen deliberately because it does not share its construction with $M$, which guards against circularity between the importance score and the evaluation. Accuracy is measured on GSM8K, StrategyQA, and the downstream task. As an out-of-distribution check, we run the SciAux misleading-context split (billhdzhao/SciAux). There, Thinking in a Crowd reports that Qwen3-8B falls from 96.22% to 77.03% with thinking enabled under two misleading snippets.
Three results would decide the question. First, at matched downstream accuracy, mediated consolidation must retain clearly more of FRIT's flip rate and early-answering area than standard EWC and shuffled-$M$. Otherwise the core claim fails. Second, the gain must appear under the independent unlearning-based measure and not only under the distractor measure used to build $M$. Third, high-$M$ weights should concentrate more in the second feed-forward projections or receiver heads than high-$F$ weights. A layer-restricted penalty on those regions should recover most of the full effect. If it does, mediated importance is both a cheaper form of protection and a map of where faithful reasoning is stored in the weights.
References
- Overcoming catastrophic forgetting in neural networks
- Elastic weight consolidation for better bias inoculation
- FRIT: Using Causal Importance to Improve Chain-of-Thought Faithfulness
- How Likely Do LLMs with CoT Mimic Human Reasoning?
- Measuring Faithfulness of Chains of Thought by Unlearning Reasoning Steps
- Thought Anchors: Which LLM Reasoning Steps Matter?
- On the Hardness of Faithful Chain-of-Thought Reasoning in Large Language Models
- Thinking in a Crowd: How Auxiliary Information Shapes LLM Reasoning