Abstract
Modern NLP models often operate as "black boxes," providing accurate predictions without explaining their underlying logic. Rationale learning aims to extract human-readable text snippets that justify these predictions. However, existing methods frequently produce disjointed "word salads" or fail to capture critical context like negations. This paper introduces SHARE (Shapley-value Guided Rationale Editor), an unsupervised framework that leverages Shapley values to model word interactions and a dedicated Rationale Editor to ensure logical flow. SHARE produces explanations that are more accurate, informative, and readable than previous state-of-the-art methods, outperforming even large generative models in extractive quality.
Introduction
The lack of interpretability in deep learning models remains a significant barrier to their deployment in sensitive domains. Rationale extraction—selecting a subset of input tokens to explain a model's output—is a promising solution. Yet, current extractive models often struggle with two primary issues: the "Word Salad" problem, where selected words are non-contiguous and incoherent, and context blindness, where functional words (e.g., "not") are ignored despite their impact on meaning.
A high-quality rationale must satisfy three criteria: sufficiency (it supports the prediction), informativeness (it contains the most relevant content), and readability (it is coherent to humans).
Methodology: The SHARE Framework
SHARE addresses these challenges through a three-phase pipeline designed to balance predictive power with human readability.
1. Rationale Extractor
Unlike traditional methods that analyze tokens independently, SHARE utilizes connected Shapley values. Derived from game theory, this approach calculates the marginal contribution of a word within its local neighborhood. This allows the model to identify word interactions, such as recognizing that "not" and "heartwarming" together provide a strong negative signal, even if "not" is neutral in isolation.
2. Rationale Editor
The Rationale Editor is a "plug-and-play" component that iteratively refines the extracted text. It employs four operations—Insertion, Deletion, Replacement, and No-operation—to optimize the rationale. A "benefit estimator" guides these edits by balancing word informativeness against "token distance," a metric that penalizes disjointed selections and encourages the formation of continuous, readable phrases.
3. Joint Learning Objective
The model is trained using a composite loss function:
- Sufficiency Loss: Ensures the rationale alone is enough for the classifier to reach the correct prediction.
- Informativeness Loss: Maximizes the distinction between rationale tokens and background noise.
- Readability Loss: Measures performance degradation when the rationale's word order is shuffled; a significant drop indicates the model is successfully utilizing the text's logical structure.
Experiments and Results
SHARE was evaluated on three benchmarks: VitaminC (claim verification), Movie Review (sentiment analysis), and MultiRC (multi-sentence reasoning).
Key Findings
- Superior Accuracy: On the Movie Review task, SHARE achieved a prediction accuracy of 0.894 using only the extracted rationale, significantly outperforming the previous state-of-the-art baseline, VCU (0.769).
- Enhanced Continuity: SHARE drastically reduced the "Token Distance" (TD). In the VitaminC dataset, SHARE achieved a TD of 0.131, compared to approximately 0.400 for other Shapley-based methods, indicating far more cohesive explanations.
- Outperforming Generative LLMs: In the MultiRC task, SHARE (using a T5 backbone) showed a 65.5% relative improvement in rationale quality over LLaMA2-7B. While LLMs often hallucinate or provide empty outputs in unsupervised settings, SHARE's extractive nature ensures groundedness.
Conclusion
By prioritizing word interactions and phrase continuity, SHARE bridges the gap between model performance and human interpretability. Its extractive approach avoids the hallucinations common in generative models while providing explanations that are both logically sound and easy to read. Future work will explore scaling SHARE to larger models and multi-modal applications.
Summary updated