Abstract: Large language models (LLMs) suffer from serious unfaithful chain-of-thought (CoT) issues. Previous work attempts to measure and explain it but lacks in-depth analysis within CoTs and does not consider the interactions among all reasoning components jointly. In this paper, we first study the CoT faithfulness issue at the granularity of CoT steps, identify two reasoning paradigms: centralized reasoning and distributed reasoning, and find their relationship with faithfulness. Subsequently, we conduct a joint analysis of the causal relevance among the context, CoT, and answer during reasoning. The result proves that, when the LLM predicts answers, it can recall correct information missing in the CoT from the context, leading to unfaithfulness issues. Finally, we propose the inferential bridging method to mitigate this issue, in which we use the attribution method to recall information as hints for CoT generation and filter out noisy CoTs based on their semantic consistency and attribution scores. Extensive experiments demonstrate that our approach effectively alleviates the unfaithful CoT problem.
Introduction
Chain-of-Thought (CoT) prompting has transformed how we interact with Large Language Models (LLMs), enabling them to tackle complex math and logic problems by "thinking out loud." However, CoT is notoriously inconsistent—sometimes it boosts performance, and other times it provides the right answer despite hallucinating a completely nonsensical reasoning path. This "faithfulness" gap makes LLMs unreliable for high-stakes decision-making. This research provides a rigorous post-mortem of CoT, moving beyond simply measuring accuracy to ask: What actually makes a reasoning chain effective, and why do models often ignore their own logic?
CoT Effectiveness and Problem Difficulty
A primary finding is that CoT utility is highly dependent on task difficulty. CoT acts as a "hard problem" specialist, providing substantial gains in complex mathematical or logical tasks, while its impact on simpler commonsense tasks is often negligible or even negative. In tasks like WinoGrande, where the solution often relies on internal knowledge retrieval rather than multi-step deduction, the overhead of generating a reasoning chain can introduce noise.
As shown in Figure 2, the performance gap between Direct and CoT prompting widens as difficulty increases in GSM8k. Conversely, for WinoGrande, the benefit is less pronounced, highlighting that CoT's effectiveness scales with the inherent complexity of the reasoning required.
The Mechanics of Unfaithfulness: Information Flow
To decode the patterns influencing these behaviors, the researchers employed two primary analytical lenses: Information Gain (IG), measuring how much "new" information a CoT provides, and Information Flow, using Integrated Gradient Attribution to trace which input tokens influence the final answer.
The most striking discovery is the "recall" safety net. When a model generates a flawed reasoning chain, it doesn't necessarily get the answer wrong. During the final answer generation, the model "reaches back" to the original question context to find the missing information it dropped during the CoT phase. It essentially "bridges" over its own bad logic, leading to unfaithful but correct results.
Figure 6 illustrates how attribution to the context changes throughout the reasoning process. For CoT to be effective, the information flow from the reasoning chain to the answer should ideally be monotonic. If the model's attention to the CoT remains flat or decreases, the CoT is likely being bypassed in favor of direct context retrieval.
Proposed Solution: The QUIRE Algorithm
To mitigate these issues, the authors developed the inferential bridging method, implemented as the QUIRE (QUestion Information Recall and Enhancement) algorithm. QUIRE works in three steps:
- AAE Recall: It first identifies which specific parts of the question context are most relevant to the solution using attribution scores.
- Hint Injection: It re-prompts the model, injecting these "recalled" context statements as hints to generate a more grounded CoT.
- IG Vote: Instead of simple majority voting, it weights the final answer based on the Information Gain of the reasoning chain, favoring CoTs that derive more logic from the question rather than hallucinating.
Key Results and Conclusion
The study evaluated four advanced models (Mistral-7B, Gemma2-9B, Llama3.1-8B, and Qwen2.5-14B) across nine datasets. QUIRE consistently outperformed standard CoT and Self-Consistency (SC) baselines:
- Accuracy Boost: Improved accuracy by up to 2.4% on the ProofWriter logic dataset.
- Faithfulness Improvement: Showed a 5.6% improvement in faithfulness on ProntoQA using the Faithful BertScore (FBS) metric.
The results prove that forcing the model to be more "faithful" (by ensuring it uses the right context) naturally leads to better "effectiveness" (getting the right answer). While currently requiring internal gradient access, future work aims to develop gradient-free methods to apply these insights to closed-source "black-box" models.
Summary updated