TL;DR
This paper introduces GrammarSHAP, a novel method for explaining NLP model predictions by incorporating sentence structure. Unlike previous word-level explainers, GrammarSHAP generates hierarchical explanations that attribute importance to phrases and clauses, offering a more intuitive understanding of how models process language. This approach significantly speeds up explanations by using masking tokens instead of a background dataset, making it more practical for complex NLP tasks.
What Problem Does This Paper Address?
Deep learning models, particularly transformers, have achieved remarkable success in Natural Language Processing (NLP). However, their immense complexity makes it difficult to understand why they make specific predictions. Existing explanation methods, like feature attribution, often focus on individual words, assigning them importance scores. This is problematic because human language is inherently structured; the meaning and sentiment of a word heavily depend on its context within phrases, clauses, and the overall sentence. Simply looking at word importance can be misleading, as it fails to capture these crucial linguistic relationships. The paper aims to bridge this gap by developing an explainer that respects and leverages the hierarchical structure of sentences to provide more meaningful and granular explanations.
Methods
The core contribution of this paper is GrammarSHAP, an extension of the SHAP (SHapley Additive exPlanations) framework. Here's how it works:
- Constituency Parsing for Token Grouping: GrammarSHAP first employs a constituency parser (specifically, the Berkeley Neural Parser) to break down sentences into their grammatical constituents (phrases, clauses, etc.) at various hierarchical levels. This allows the method to group related words into meaningful multi-word tokens.
- Hierarchical Explanations: By iterating through the parsed sentence structure, GrammarSHAP generates explanations at multiple levels of granularity. This means it can attribute importance not just to individual words but also to phrases like "the plot development is disjointed and slow" or even larger clauses.
- Efficient KernelSHAP Adaptation: The method adapts the KernelSHAP algorithm, a model-agnostic explainer. The key innovation for efficiency is replacing the traditional SHAP background dataset with masking tokens (like
[MASK]). This significantly speeds up the computation by avoiding the need to query the model with numerous background samples. This also helps mitigate artifacts caused by special tokens like[SEP]. - Additive KernelSHAP: For comparison, the authors also introduce "Additive KernelSHAP," which combines SHAP values from individual tokens within a group to represent the group's importance, leveraging the constituency tree structure.
Datasets
The paper evaluates GrammarSHAP on two standard NLP datasets for sentiment analysis:
- IMDb Movie Reviews: A dataset of movie reviews, commonly used for binary sentiment classification.
- SST-2 (Stanford Sentiment Treebank): Another dataset for sentiment analysis, known for its fine-grained annotations.
The models used were versions of DistilBERT pre-trained on these datasets, achieving accuracies of 93.7% (IMDb) and 91.3% (SST-2).
Key Insights and Ideas
- Language Structure Matters for Explanations: The most significant insight is that simply attributing importance to individual words is insufficient for understanding complex NLP models. Language is hierarchical, and explanations should reflect this to be truly informative.
- Hierarchical Explanations Reveal Nuance: GrammarSHAP's ability to provide explanations at different levels of granularity allows for the identification of both positive and negative contributions within a single sentence, even when the overall sentiment is clear. This is crucial for understanding how models handle mixed-sentiment text.
- Masking Tokens are a Game-Changer for Efficiency: Replacing background datasets with
[MASK]tokens is a clever optimization that drastically reduces computation time without sacrificing explanation quality, making hierarchical explanations more feasible. - Beyond Word-Level is Necessary: The paper strongly advocates for moving beyond word-level explanations to capture the richer semantic and syntactic relationships that models exploit.
Key Results
- Efficiency: GrammarSHAP, by using masking tokens, achieves a significant speedup. While not as fast as PartitionSHAP (which is described as "coarse"), it is substantially faster than a custom Additive KernelSHAP implementation, with running times in the order of minutes rather than hours for a sample of 20 texts.
- Explanation Quality: Qualitative analysis (shown in Figures 3, 4, 5, and 6) demonstrates that GrammarSHAP can identify contributions at various hierarchical levels, including those that contradict the overall sentiment. This is a key advantage over methods like PartitionSHAP, which are described as "coarse," and Additive KernelSHAP, which struggles to identify opposing sentiments.
- Mitigation of Artifacts: The use of masking tokens effectively removes artifacts related to special tokens like
[SEP], which can otherwise inflate their importance in explanations.
Limitations and Future Work
- Quantitative Evaluation: The authors acknowledge that their evaluation is primarily qualitative. They suggest that future work should focus on developing and applying quantitative diagnostic properties and word-level metrics to rigorously assess explanation quality.
- Execution Time for Real-Time Applications: While GrammarSHAP is reasonably efficient, further improvements in speed are needed for real-time explanation generation, especially for very large-scale applications.
- Focus on Negative Sentiment: The paper notes that their method, like some others, tends to highlight contributions that align with the overall sentiment. Future work could explore methods to better capture explanations that contradict the predicted sentiment.
- Broader Application: The authors plan to focus on quantitative evaluation as a priority for future work.
Summary updated