Self-interpretable Convolutional Neural Networks for Text Classification

Wei Zhao, Rahul Singh, Tarun Joshi, Agus Sudjianto, Vijayan N. Nair

Abstract: Deep learning models for natural language processing (NLP) are inherently complex and often viewed as black box in nature. This paper develops an approach for interpreting convolutional neural networks for text classification problems by exploiting the local-linear models inherent in ReLU-DNNs. The CNN model combines the word embedding through convolutional layers, filters them using max-pooling, and optimizes using a ReLU-DNN for classification. To get an overall self-interpretable model, the system of local linear models from the ReLU DNN are mapped back through the max-pool filter to the appropriate n-grams. Our results on experimental datasets demonstrate that our proposed technique produce parsimonious models that are self-interpretable and have comparable performance with respect to a more complex CNN model. We also study the impact of the complexity of the convolutional layers and the classification layers on the model performance.

Introduction

Deep Neural Networks (DNNs) have become the state-of-the-art for NLP tasks such as sentiment analysis. However, their "black box" nature poses significant challenges in regulated sectors like banking and finance, where model decisions must be explainable to regulators and customers. While post-hoc explanation methods like LIME or SHAP exist, they often rely on approximations. This research proposes a "self-interpretable" CNN architecture that provides human-readable explanations by design, mapping classification decisions directly back to specific n-grams in the source text.

Methodology

The proposed framework follows a multi-step process to bridge the gap between high-performance deep learning and linear model interpretability.

Model Architecture

The system uses a standard CNN structure for feature engineering. Input text is converted into an embedding matrix where each row corresponds to a word vector. Convolutional filters of varying sizes (capturing uni-grams, bi-grams, and tri-grams) are applied, followed by a max-pooling layer. The final classification is handled by a ReLU-based DNN.

Architecture of the self-interpretable CNN model showing the transition from word embeddings to classification

Exploiting Local Linearity

A key mathematical insight of this work is that a ReLU-DNN partitions the input space into multiple sub-regions. Within each region, the model behaves as a Local Linear Model (LLM). By calculating the weights of these LLMs, the researchers can determine feature importance directly from the model's internal state.

ReLU DNN architecture and its corresponding input space partitions into local linear regions

Simplification and Mapping

To prevent an overwhelming number of sub-regions, the authors apply L1 regularization and a merging algorithm called Aletheia. This clusters similar linear regions together, drastically reducing complexity. Finally, the weights from the simplified linear classifier are mapped back through the max-pool filters to the original n-grams, allowing users to see exactly which words (e.g., "highly recommended") drove the prediction.

Experimental Results

The model was evaluated using the Yelp Review dataset (20,000 training and 10,000 testing samples). Reviews were standardized to a length of 800 tokens.

Performance and Interpretability

  • Accuracy: The self-interpretable model achieved a test accuracy of 90.67% with 6 regions, and reached 90.84% when simplified further.
  • Feature Identification: The model successfully identified intuitive sentiment drivers. Positive predictions were linked to filters for "scrumptious" and "is amazing," while negative predictions were triggered by terms like "horrific service."
  • Sparsity: L1 regularization reduced the number of local linear regions from 621 to 25 with negligible loss in accuracy, demonstrating that complex DNNs can often be simplified into a few representative linear models.

Discussion and Key Insights

The study revealed that "filter complexity" (the number of convolutional filters) has a more significant impact on accuracy than the complexity of the ReLU classifier itself. For the Yelp dataset, the merging algorithm showed that the complex ReLU DNN could eventually be simplified into a single linear region—essentially a Logistic Regression on top of CNN features—without sacrificing performance.

Conclusion

This approach ensures that the "interpretation" of a model is mathematically identical to its "computation." Future work will explore whether more complex datasets require the multi-region capabilities of the ReLU-DNN or if the convergence to simplicity holds across broader domains. Additionally, the researchers suggest investigating how these interpretable features might be utilized in transfer learning scenarios.

Summary updated

Abstract

Deep learning models for natural language processing (NLP) are inherently complex and often viewed as black box in nature. This paper develops an approach for interpreting convolutional neural networks for text classification problems by exploiting the local-linear models inherent in ReLU-DNNs. The CNN model combines the word embedding through convolutional layers, filters them using max-pooling, and optimizes using a ReLU-DNN for classification. To get an overall self-interpretable model, the system of local linear models from the ReLU DNN are mapped back through the max-pool filter to the appropriate n-grams. Our results on experimental datasets demonstrate that our proposed technique produce parsimonious models that are self-interpretable and have comparable performance with respect to a more complex CNN model. We also study the impact of the complexity of the convolutional layers and the classification layers on the model performance.
BibTeX
@misc{zhao2021self,
  author = {Wei Zhao and Rahul Singh and Tarun Joshi and Agus Sudjianto and Vijayan N. Nair},
  title = {{Self-interpretable Convolutional Neural Networks for Text Classification}},
  year = {2021},
  eprint = {2105.08589},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2105.08589},
}
Figure