TL;DR
This paper introduces ConRAT, a novel self-interpretable model that explains predictions by extracting and aggregating "concepts" (text snippets) rather than relying on a single rationale. ConRAT achieves superior predictive and explanation performance by learning interpretable, diverse, and non-overlapping concepts, outperforming existing methods on sentiment classification tasks.
What Problem Does This Paper Address?
The increasing adoption of neural networks in decision-critical fields is hampered by their "black-box" nature, making it difficult to understand why a prediction was made. While "rationalization" methods aim to address this by selecting relevant input features (like text snippets), a single selected snippet often fails to capture the multi-faceted nature of human reasoning. Humans typically weigh multiple aspects or "concepts" when making complex decisions. Existing rationalization methods struggle to represent this multi-faceted explanation, often producing noisy or incomplete rationales. This paper tackles the challenge of creating models that not only predict accurately but also provide explanations that are more aligned with human cognitive processes, specifically by leveraging the idea of concepts.
Methods
The core of the proposed solution is the ConRAT (Concept-based Rationalizer) model. ConRAT is a self-explaining neural network that operates in three main stages:
- Concept Generator: This module identifies potential "concepts" within the input text. A concept is defined as a sequence of consecutive words, extracted using a binary mask. The generator is trained to produce these masks.
- Concept Selector: Since not all generated concepts might be relevant for a specific input document, this module learns to detect which of the K concepts are actually present. It uses an attention mechanism to create a concept representation and then a sigmoid function to output a presence score (0 or 1) for each concept.
- Predictor: This module takes the selected concepts and their presence scores to compute the final prediction. It aggregates the predictions derived from each selected concept using a linear combination, where weights represent the contribution of each concept to the final outcome.
To ensure the learned concepts are interpretable and useful, ConRAT incorporates two key regularizers:
- Overlap Regularizer ($L_{overlap}$): This penalizes redundancy between concept representations, encouraging diversity.
- Diversity Regularizer ($L_{div}$): This further promotes diversity by minimizing the cosine similarity between concept representations.
Additionally, the authors propose two optional techniques to further enhance performance:
- Knowledge Distillation: Training ConRAT using the outputs of a pre-trained "teacher" model.
- Concept Pruning: Selecting the most dissimilar concepts at inference time to improve interpretability.
Datasets
The paper evaluates ConRAT on two datasets:
- Amazon Electronics Dataset: A single-aspect binary classification dataset. The authors followed a filtering process to select reviews with evidence for both positive and negative sentiments, creating a balanced dataset of 24,000 samples.
- Multi-aspect Beer Reviews Dataset: This dataset, commonly used in rationalization research, contains reviews with multiple aspects (Appearance, Aroma, Palate, Taste, Overall). For ConRAT's evaluation, only the overall rating was used for training, while baselines were trained separately on each aspect. The dataset was binarized and sampled to create 60,000 balanced examples.
Key Insights and Ideas
- Human reasoning is concept-driven: The core idea is that human explanations often rely on multiple concepts, not just a single rationale. ConRAT aims to mimic this by learning and aggregating multiple concepts.
- Self-interpretable models are feasible: ConRAT is designed to be inherently interpretable, generating explanations as part of its prediction process, rather than relying on post-hoc methods.
- Unsupervised concept discovery is powerful: ConRAT learns concepts in an unsupervised manner, demonstrating that meaningful and human-understandable concepts can be extracted without explicit human annotations for each concept.
- Multi-faceted explanations outperform single rationales: By aggregating multiple concepts, ConRAT provides a more complete and nuanced explanation compared to traditional single-rationale methods.
- Regularization is key for interpretable concepts: The proposed overlap and diversity regularizers are crucial for guiding ConRAT to learn concepts that are not only relevant but also distinct and understandable.
Key Results
- Superior performance on sentiment classification: ConRAT significantly outperforms state-of-the-art baselines on both single- and multi-aspect sentiment classification tasks, even when trained only on the overall label.
- Human-aligned rationales: Experiments show that ConRAT's generated concepts align well with human-defined aspects and rationalizations, as validated through both objective metrics and subjective human evaluations.
- Accuracy gains: On the Amazon Electronics dataset, ConRAT achieved an accuracy of 75.5% with one concept (ConRAT-1), surpassing baselines. For the multi-aspect beer reviews, ConRAT achieved the best overall accuracy (91.4%) and aspect-specific accuracies, outperforming baselines trained on individual aspects.
- Impact of concept count: Increasing the number of concepts (K) generally boosted performance, with K=10 and K=20 showing improvements, though interpretability might decrease beyond a certain point.
- Ablation studies confirm module importance: Removing the overlap or diversity regularizers led to significant drops in rationale performance, highlighting their importance for learning interpretable concepts.
Limitations and Future Work
- Counterfactual Rationales: While ConRAT excels at finding factual rationales, it's not explicitly designed to infer counterfactual concepts. The authors suggest that increasing the number of concepts and using pruning might help in this regard, but it's not a guaranteed outcome.
- Interpretability vs. Concept Count: The paper notes that interpretability can follow a "bell curve" with respect to the number of concepts, suggesting that too many concepts might lead to less human-understandable explanations, potentially due to expected overlaps between discriminative concepts.
- Spurious Correlations: While ConRAT mitigates spurious correlations by considering multiple concepts, the paper acknowledges that concepts are not guaranteed to be free from them, especially without explicit signals for grounding.
- Future Work: The authors suggest exploring how to explicitly infer counterfactual concepts and further investigating the trade-offs between concept count, performance, and interpretability. They also mention that the method is adaptable for regression problems.
Summary updated