Improving Interpretability via Explicit Word Interaction Graph Layer

Arshdeep Sekhon, Hanjie Chen, Aman Shrivastava, Zhe Wang, Yangfeng Ji, Yanjun Qi

Abstract

Recent NLP literature has seen growing interest in improving model interpretability. Along this direction, we propose a trainable neural network layer that learns a global interaction graph between words and then selects more informative words using the learned word interactions. Our layer, WIGRAPH, can plug into any neural network-based NLP text classifiers right after its word embedding layer. Across multiple SOTA NLP models and various NLP datasets, we demonstrate that adding the WIGRAPH layer substantially improves NLP models' interpretability and enhances models' prediction performance at the same time.

Introduction

Deep learning models for Natural Language Processing (NLP) are notoriously "black boxes." While post-hoc explanation tools like LIME or SHAP provide insights into model decisions (e.g., identifying "good" as a reason for a "positive" sentiment label), they do not alter the underlying decision-making process. Furthermore, existing "intrinsic interpretability" methods often analyze words in isolation, ignoring the linguistic reality that context and word interactions are paramount. For instance, the word "different" is neutral on its own, but "refreshingly different" conveys a strong positive sentiment.

To address these limitations, we introduce WIGRAPH (Word Interaction Graph), a plug-and-play neural network layer designed to improve both accuracy and interpretability by explicitly modeling word interactions. By building a global interaction graph directly into the architecture, WIGRAPH forces the model to rely on meaningful relationships between words to make predictions.

The WIGRAPH Architecture

WIGRAPH is designed to be inserted into standard NLP classifiers (such as BERT, RoBERTa, or LSTMs) immediately following the word embedding stage. The layer consists of three primary components:

  1. Stochastic Edge Discovery: The layer learns a global, undirected graph where each word in the vocabulary serves as a node. It discovers edges—representing interactions—that are statistically significant for the specific classification task (e.g., sentiment analysis).
  2. Neural Message Passing: Utilizing a modified Graph Convolutional Network (GCN) approach, the layer updates word representations by "listening" to neighbors in the learned interaction graph. This allows the model to incorporate contextual dependencies early in the processing pipeline.
  3. Variational Information Bottleneck (VIB-WI) Loss: To ensure the model does not simply memorize all word pairs, an Information Bottleneck objective is applied. This constraint forces the model to be "maximally compressive," retaining only the interactions that are truly predictive of the final label while discarding irrelevant noise.

Experimental Setup

The effectiveness of WIGRAPH was evaluated across six diverse text classification benchmarks:

  • Sentiment Analysis: SST-1, SST-2, and IMDB.
  • Topic Classification: AG News.
  • Question Type Classification: TREC.
  • Subjectivity Detection: Subj.

Additionally, the researchers extended the concept to computer vision using the CUB (Caltech-UCSD Birds-200-2011) dataset, modeling interactions between high-level visual concepts like "blue forehead" and "cone-shaped bill" to demonstrate the layer's versatility.

Results and Discussion

WIGRAPH demonstrates a rare "win-win" in machine learning: simultaneous improvements in both performance and interpretability.

Prediction Performance

Across 24 combinations of models (including LSTM, BERT, and DistilBERT) and datasets, WIGRAPH consistently outperformed base models. Notably, it provided a 2.63% accuracy boost to an LSTM on the SST-1 dataset. In the CUB vision task, adding WIGRAPH to a concept bottleneck model improved accuracy from 80.04% to 95.74% when human interventions were applied.

Interpretability and Faithfulness

Interpretability was measured using the Area Over Perturbation Curve (AOPC), which quantifies the drop in model confidence when "important" words (as identified by the model) are removed. WIGRAPH significantly outperformed both standard base models and previous state-of-the-art interpretability layers like VMASK. By forcing the model to use an explicit interaction graph, the highlighted words align more closely with human intuition and linguistic context.

Conclusion and Future Work

WIGRAPH provides a robust framework for making NLP models intrinsically interpretable without sacrificing performance. While current implementations are limited by the scalability of the global interaction matrix—requiring vocabulary filtering for very large sets—future work aims to address these computational constraints. Additionally, researchers plan to explore higher-order interactions beyond word pairs and apply the architecture to complex cross-sentence tasks like Natural Language Inference (NLI).

Summary updated

Abstract

Recent NLP literature has seen growing interest in improving model interpretability. Along this direction, we propose a trainable neural network layer that learns a global interaction graph between words and then selects more informative words using the learned word interactions. Our layer, we call WIGRAPH, can plug into any neural network-based NLP text classifiers right after its word embedding layer. Across multiple SOTA NLP models and various NLP datasets, we demonstrate that adding the WIGRAPH layer substantially improves NLP models' interpretability and enhances models' prediction performance at the same time.
BibTeX
@inproceedings{sekhon2023improving,
  author = {Arshdeep Sekhon and Hanjie Chen and Aman Shrivastava and Zhe Wang and Yangfeng Ji and Yanjun Qi},
  title = {{Improving Interpretability via Explicit Word Interaction Graph Layer}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2023},
  url = {https://openreview.net/forum?id=8M7dUhW9EZ},
}
Figure