Introduction
Modern Natural Language Processing (NLP) is dominated by Transformer-based models. While these models achieve remarkable accuracy, they often function as "black boxes," providing little insight into the reasoning behind their predictions. This lack of transparency is particularly problematic in sensitive domains or complex tasks like sarcasm detection, where meaning is derived from context and incongruity rather than individual word importance. This research addresses the "interpretability tax"—the common trade-off where transparent models underperform compared to their opaque counterparts—by proposing a framework that integrates Prototypical Networks with high-performance Language Model (LM) encoders.
Methodology
The proposed framework shifts the classification paradigm from direct mapping to similarity-based reasoning. Instead of a standard classification head, the model uses a "white-box" architecture where decisions are based on similarity scores to learned prototypes—representative examples within the vector space.
The core components of this approach include:
- GAProtoNet (Graph-attention Enhanced Prototype Network): Utilizing Multi-head Graph Attention (GAT) instead of simple distance metrics, the model selectively attends to specific semantic aspects of the input relative to neighboring prototypes.
- Multi-View Sarcasm Detection: To capture the nuance of sarcasm, the model combines semantic and sentiment embeddings, employing a novel incongruity loss to identify clashes between implicit and explicit sentiments.
- Contrastive Learning for GNNs: For document classification tasks, Graph Contrastive Learning (GraphCL) is used to distill salient structural patterns into prototypes, ensuring they represent meaningful real-world training cases.
Experimental Evaluation
The methods were evaluated across three primary domains: general text classification (binary to ten-label tasks), contextual sarcasm detection, and graph-based document classification (citations and co-authorships).
Key Results
The research demonstrates that GAProtoNet achieves state-of-the-art performance among prototype-based models. Crucially, it remains within 0.3% accuracy of original "black-box" Transformer models, effectively eliminating the interpretability tax. In sarcasm detection, the multi-view framework established new state-of-the-art benchmarks. Furthermore, the model proved robust, maintaining high interpretability (90%-95% distinguished prototypes) even when varying the number of prototypes from 10 to 40.
Discussion and Insights
A significant advantage of this approach is the move toward sentence-level explanations. Unlike traditional word-level attention maps, which can be noisy or misleading, sentence-level prototypes provide intuitive justifications: a sentence is classified as "Sarcastic" because it resembles a known sarcastic example. Visualization via t-SNE confirms that learned prototypes are evenly distributed, efficiently spanning the semantic space without redundancy.
Conclusion and Future Work
This work proves that "white-box" models can match the performance of "black-box" counterparts. While successful in text, training graph-based prototypes remains challenging due to structural complexity. Future work aims to refine the integration of contrastive learning with Graph Neural Networks (GNNs) and explore the scalability of prototype networks for datasets with high-cardinality label sets.
Summary updated