TL;DR
Large Language Models (LLMs) can generate explanations for their predictions, but these explanations often don't accurately reflect the model's true reasoning. This paper introduces SR-NLE, a framework that allows LLMs to iteratively critique and refine their own explanations without human supervision, significantly improving their faithfulness.
What Problem Does This Paper Address?
As LLMs become more powerful and are deployed in critical applications, understanding why they make certain decisions is crucial for trust and accountability. Natural Language Explanations (NLEs) are a promising way to provide these justifications. However, a significant challenge is ensuring that these NLEs are "faithful" – meaning they accurately represent the model's internal reasoning process. Existing methods often require architectural changes or extensive fine-tuning. This paper explores whether LLMs can achieve this faithfulness improvement through self-critique and refinement, a capability that has shown promise in other LLM output generation tasks.
Methods
The core of the SR-NLE framework is an iterative process of self-critique and refinement. For a given input, the LLM first generates an answer and then an initial explanation. This explanation then goes through multiple rounds of refinement. The key innovation lies in the feedback mechanisms used to guide this refinement:
- Natural Language Feedback (NLF): The LLM generates free-form textual self-critiques of its own explanation.
- Important Word Feedback (IWF): This novel approach uses feature attribution methods to identify the most important words in the input that contributed to the prediction. This feedback highlights which parts of the input the explanation should focus on. IWF has two variants:
- Prompt-based (IWF-Pmt): The LLM is prompted to directly assign importance scores to input words.
- Attribution-based (IWF-Attn): This uses established attribution techniques (like Integrated Gradients or attention weights) to quantify word importance.
After receiving feedback (either NLF or IWF), the LLM refines its explanation. This cycle can be repeated multiple times to incrementally improve faithfulness.
Datasets
The authors evaluated their framework on three widely used natural language reasoning datasets with NLEs:
- ComVE: Identifying common sense violations between two sentences.
- ECQA: Answering multiple-choice common sense questions.
- e-SNLI: Determining the logical relationship (contradiction, neutral, entailment) between a premise and a hypothesis.
They used the first 1,000 instances from each dataset's test set.
Key Insights and Ideas
- LLMs can self-critique for faithfulness: The most significant insight is that LLMs possess an inherent capability to assess and improve the faithfulness of their own explanations, given appropriate guidance. This bypasses the need for external supervision or task-specific training.
- Important Word Feedback is powerful: The IWF mechanism, particularly the attribution-based variant, consistently outperformed NLF. This suggests that pinpointing specific influential input words provides a more direct and effective signal for improving explanation faithfulness than general textual critiques.
- Refinement is most impactful early on: The largest reduction in unfaithfulness occurs during the first refinement round, indicating that initial critiques are highly effective. Subsequent rounds offer diminishing returns, suggesting that 2-3 rounds might be an optimal trade-off between performance and computational cost.
- Faithfulness and length are inversely related: As explanations become longer through refinement, their faithfulness generally increases. However, IWF methods achieve greater faithfulness improvements for a given increase in length compared to NLF, highlighting their efficiency.
Key Results
The SR-NLE framework demonstrated significant improvements in explanation faithfulness:
- The best performing method (IWF-Attn) achieved an average unfaithfulness rate of 36.02%, a substantial reduction from the initial NLEs (54.81%), representing an absolute reduction of 18.79%.
- SR-NLE consistently outperformed baseline methods like "Init-NLE" (initial explanations) and "SC-NLE" (self-consistency).
- IWF variants consistently outperformed NLF, with IWF-Attn, IWF-IG, and IWF-Pmt achieving lower unfaithfulness rates than NLF by an average of 11.56%, 11.28%, and 10.35%, respectively.
- The framework showed robust performance across different LLMs (Falcon, Llama, Mistral, Qwen) and datasets.
Limitations and Future Work
- Explanation Paradigm: The current framework focuses on post-hoc NLEs. It's unclear if it generalizes to other explanation paradigms like chain-of-thought or jointly generated rationales.
- Evaluation Method: The study relies solely on counterfactual tests for faithfulness. Future work could incorporate other tests (consistency, simulatability) and consider aspects like plausibility and completeness.
- Attribution Method Reliability: The effectiveness of IWF-Attn depends on the reliability of underlying attribution techniques, which can be sensitive to implementation details and computational costs.
- Model Scale: Experiments were conducted on LLMs under 10B parameters. Further investigation is needed to understand how model scale affects SR-NLE's performance.
The authors suggest future work could explore additional feedback mechanisms and apply SR-NLE to a broader range of domains and reasoning tasks.
Summary updated