Introduction
Deep neural networks have achieved state-of-the-art results in text classification, yet they remain notoriously opaque "black boxes." In high-stakes fields like medicine, the rationale behind a prediction is often as critical as the prediction itself. Previous attempts at self-explaining models typically relied on single-word highlights or fixed-length n-grams, both of which struggle to capture the full context of complex phrases (e.g., "bleeding" vs. "bleeding after nasogastric tube insertion").
Methodology: Multi-Granular Forest Structures
The researchers introduced a Multi-granular Text Encoder that organizes all possible n-grams into hierarchical structures rather than flat sequences. They explored several configurations, including standard trees, pyramids, and branching forests.
The "forest" structures are particularly effective as they allow the model to represent every possible n-gram while grouping those with shared prefixes or suffixes. By utilizing a tree-structured LSTM, the model reuses representations of smaller n-grams to construct larger ones, significantly improving computational efficiency. An attention mechanism then identifies the most influential multi-word units to serve as the model's explanation.
Experimental Results
The model was tested on a medical abstract classification task involving five disease categories. The proposed BiForest model (combining left and right-branching forests) achieved 65.2% accuracy, surpassing BiLSTM (62.7%) and CNN (62.5%) baselines.
Beyond accuracy, the model demonstrated superior efficiency. As shown in the performance table, the LeftForest variant is roughly 2x faster than a CNN and 3x faster than a BiLSTM during training, while requiring a fraction of the parameters. This efficiency stems from the parameter sharing inherent in the tree-structured LSTM approach.
Explainability and Sufficiency
Qualitative analysis showed the model successfully identifies complex medical phrases as evidence. Quantitatively, the researchers found that feeding only the top five phrases identified by the model into a simple classifier yielded accuracy nearly identical to using the entire original text. This confirms that the extracted multi-granular units are truly "sufficient" for the classification task.
Conclusion
This work presents a robust framework for self-explaining text categorization. By moving beyond single-word granularity to hierarchical forest structures, the model provides human-readable justifications without sacrificing performance or efficiency. Future work may explore extending the n-gram limit beyond the current cap of nine and testing the approach in other domains like sentiment analysis.
Summary updated