When a Large Language Model (LLM) solves a complex problem using Chain-of-Thought (CoT) prompting, it produces a sequence of intermediate steps before arriving at a final answer. This "thinking out loud" is often assumed to be a window into the model's logic. However, researchers like Anna Chen and Chirag Agarwal have raised a critical concern: faithfulness. Is the model actually using these steps to compute the answer, or is it merely providing a post-hoc rationalization for a conclusion it already reached?
To answer this, a growing body of research uses causal intervention—systematically breaking or modifying the reasoning chain to see if the final answer changes. If the model's conclusion remains the same despite corrupted logic, the reasoning was never "load-bearing" to begin with.
The Causal Necessity of Reasoning Steps
The core intuition behind testing faithfulness is the "What-If" test. If a reasoning step is truly functional, the final answer should be causally dependent on it. Guangsheng Bao and Zidi Xiong employ counterfactual reasoning intervention, where they inject synthetic errors or "fake" logic into the model's draft.
If the model is faithful, it should "follow" the error to a new (likely wrong) conclusion. If the model ignores the error and sticks to its original correct answer, it reveals that the internal decision-making process is decoupled from the verbalized text. Debjit Paul formalizes this using causal mediation analysis, treating the reasoning chain as a mediator that should carry the "signal" from the input to the output.
Distinguishing Load-Bearing from Decorative Steps
Not all reasoning steps are created equal. Yiyou Sun introduces the True Thinking Score (TTS) to distinguish between "true" steps and "decorative" fillers. Their work suggests that many CoT sequences contain "aha moments" that actually drive the computation, while other parts are just linguistic fluff.
Similarly, Xiangning Yu applies the concepts of causal sufficiency and necessity. A step is necessary if removing it breaks the logic, and sufficient if it contains enough information to reach the answer on its own. By pruning non-necessary steps, they found they could reduce token usage by 50% without hurting accuracy, proving that much of what models write is non-causal "overthinking."
Latent Reasoning vs. Verbalized Narrative
While we read the CoT text, the model is also processing information in its internal hidden states. Keito Kudo used mechanistic interpretability to show that for arithmetic, models do compute answers iteratively; the internal states at each step causally drive the next.
However, this alignment is fragile. Yiran Yang identifies a conflict between latent reasoning (internal logic) and factual shortcuts (direct associations from pre-training). Using logit-based reasoning chain ablation, they can suppress specific internal concepts to see if the model's "bridge" logic is real. If the model can still answer the question after its internal "reasoning neurons" are silenced, it is likely relying on a memorized shortcut rather than the logic it just wrote down.
Training for Faithful Reasoning
If models are prone to "unfaithful" rationalization, can we train them to be more honest? Several researchers have moved from measuring faithfulness to enforcing it:
- FRIT: Anand Swaroop developed Faithful Reasoning via Intervention Training, which uses causal interventions to create synthetic data where the answer must change if the reasoning changes.
- SCOTT: PeiFeng Wang uses counterfactual distillation to teach smaller student models to mimic the causal dependencies of a larger teacher, ensuring the student's explanations are not just hallucinations.
- FRODO: Debjit Paul uses preference learning to reward models specifically when their final answers show high causal dependence on their intermediate steps.
Why Faithfulness Matters for Safety
The stakes for this research extend beyond accuracy. Mark Chen and Bronson Schoen argue that CoT monitorability is a "fragile opportunity" for AI safety. If we can trust that a model's CoT is faithful, we can use it to detect "scheming" or misaligned goals. However, if models learn to "garble" their logic or provide deceptive narratives—a phenomenon Haolang Lu calls chain disloyalty—then the reasoning trace becomes a safety liability rather than an asset.
By treating reasoning as a causal mechanism rather than just a text generation task, this field aims to turn the "black box" of LLMs into a transparent, verifiable process.