Multi-Granular Text Encoding for Self-Explaining Categorization

Zhiguo Wang, Yue Zhang, Mo Yu, Wei Zhang, Lin Pan, Linfeng Song, Kun Xu, Yousef El-Kurdi

Introduction

Deep neural networks have achieved state-of-the-art results in text classification, yet they remain notoriously opaque "black boxes." In high-stakes fields like medicine, the rationale behind a prediction is often as critical as the prediction itself. Previous attempts at self-explaining models typically relied on single-word highlights or fixed-length n-grams, both of which struggle to capture the full context of complex phrases (e.g., "bleeding" vs. "bleeding after nasogastric tube insertion").

Methodology: Multi-Granular Forest Structures

The researchers introduced a Multi-granular Text Encoder that organizes all possible n-grams into hierarchical structures rather than flat sequences. They explored several configurations, including standard trees, pyramids, and branching forests.

Hierarchical structures for n-gram organization

The "forest" structures are particularly effective as they allow the model to represent every possible n-gram while grouping those with shared prefixes or suffixes. By utilizing a tree-structured LSTM, the model reuses representations of smaller n-grams to construct larger ones, significantly improving computational efficiency. An attention mechanism then identifies the most influential multi-word units to serve as the model's explanation.

Experimental Results

The model was tested on a medical abstract classification task involving five disease categories. The proposed BiForest model (combining left and right-branching forests) achieved 65.2% accuracy, surpassing BiLSTM (62.7%) and CNN (62.5%) baselines.

Performance comparison across models

Beyond accuracy, the model demonstrated superior efficiency. As shown in the performance table, the LeftForest variant is roughly 2x faster than a CNN and 3x faster than a BiLSTM during training, while requiring a fraction of the parameters. This efficiency stems from the parameter sharing inherent in the tree-structured LSTM approach.

Explainability and Sufficiency

Qualitative analysis showed the model successfully identifies complex medical phrases as evidence. Quantitatively, the researchers found that feeding only the top five phrases identified by the model into a simple classifier yielded accuracy nearly identical to using the entire original text. This confirms that the extracted multi-granular units are truly "sufficient" for the classification task.

Conclusion

This work presents a robust framework for self-explaining text categorization. By moving beyond single-word granularity to hierarchical forest structures, the model provides human-readable justifications without sacrificing performance or efficiency. Future work may explore extending the n-gram limit beyond the current cap of nine and testing the approach in other domains like sentiment analysis.

Summary updated

Abstract

Self-explaining text categorization requires a classifier to make a prediction along with supporting evidence. A popular type of evidence is sub-sequences extracted from the input text which are sufficient for the classifier to make the prediction. In this work, we define multi-granular ngrams as basic units for explanation, and organize all ngrams into a hierarchical structure, so that shorter ngrams can be reused while computing longer ngrams. We leverage a tree-structured LSTM to learn a context-independent representation for each unit via parameter sharing. Experiments on medical disease classification show that our model is more accurate, efficient and compact than BiLSTM and CNN baselines. More importantly, our model can extract intuitive multi-granular evidence to support its predictions.
BibTeX
@inproceedings{wang2019multi,
  author = {Zhiguo Wang and Yue Zhang and Mo Yu and Wei Zhang and Lin Pan and Linfeng Song and Kun Xu and Yousef El-Kurdi},
  title = {{Multi-Granular Text Encoding for Self-Explaining Categorization}},
  booktitle = {BlackboxNLP @ ACL},
  year = {2019},
  doi = {10.18653/v1/w19-4805},
}
Figure