Abstract
Deep learning models often function as "black boxes," which is problematic in high-stakes domains like medicine. This paper proposes a multi-granular text encoder that provides self-explaining classifications by highlighting evidence of varying lengths, from single words to long phrases. By organizing n-grams into hierarchical forest structures and utilizing tree-structured LSTMs, the model achieves superior accuracy and computational efficiency compared to standard CNNs and BiLSTMs.
Introduction
In fields such as clinical medicine, understanding the rationale behind a model's prediction is as critical as the prediction itself. While deep learning has advanced text categorization, existing "self-explaining" methods face several challenges. Many models only highlight single words, whereas medical meaning often resides in multi-word phrases (e.g., "petechial bleed"). Furthermore, sequential models like standard LSTMs often "smear" context across hidden states, making it difficult to pinpoint the exact source of evidence. CNN-based approaches, while better at capturing n-grams, often rely on fixed-length filters that lack the flexibility to capture natural language patterns efficiently.
Methodology
The authors introduce a Multi-Granular Text Encoder designed to extract intuitive, context-independent evidence. The core of this approach lies in the structural organization of n-grams.
Hierarchical Forest Structures
To avoid the computational redundancy of calculating every possible n-gram independently, the model organizes n-grams into hierarchical structures. The authors propose Left-branching and Right-branching forests, where shorter n-grams are reused to construct longer ones. This hierarchical reuse ensures that each sub-unit contributes uniquely to the representation, solving the "over-counting" problem found in simpler pyramid structures.
Tree-Structured LSTM and Attention
A tree-structured LSTM composition function computes node embeddings, with parameters shared across all n-gram orders. This design makes the model significantly more compact than a CNN. An attention mechanism is then applied over all nodes in the forest. The resulting attention weights ($\alpha$) directly indicate the importance of specific words or phrases, providing a transparent, human-readable explanation for the model's output.
Experimental Results
The model was evaluated on a public medical text classification dataset consisting of 14,102 medical abstracts categorized into five disease classes.
Performance and Efficiency
The BiForest model, which combines representations from both Left- and Right-branching forests, achieved an accuracy of 65.2%, outperforming BiLSTM (62.7%) and CNN (62.5%) baselines. Notably, the Forest models maintain higher accuracy as the n-gram order increases, whereas CNN performance tends to fluctuate.
Beyond accuracy, the model demonstrates significant efficiency gains. The LeftForest was nearly 3x faster than BiLSTM and 2x faster than CNN during training for n-gram orders up to 7. It is also remarkably compact, requiring only ~168k parameters compared to ~848k for the CNN baseline.
Conclusion
The multi-granular text encoder provides a robust framework for interpretable text categorization. By leveraging hierarchical forest structures, it captures nuanced evidence while remaining computationally efficient. Future work may involve exploring extremely long-range dependencies and integrating these hierarchical structures with large-scale pre-trained models to further enhance performance.
Summary updated