TextGT: A Double-View Graph Transformer on Text for Aspect-Based Sentiment Analysis
Aspect-Based Sentiment Analysis (ABSA) represents a sophisticated frontier in Natural Language Processing, requiring models to discern sentiment targeted at specific entities or "aspects" within a sentence. For instance, in the review "The food was great but the service was slow," a model must correctly assign a positive sentiment to the aspect food and a negative one to service. Achieving this level of granularity necessitates a deep understanding of both the linear sequence of words and the underlying syntactic structure—the grammatical relationships that link descriptors to their targets.
The Challenge: Over-smoothing in Deep Networks
Historically, researchers have employed Graph Neural Networks (GNNs) to process syntactic dependency trees and Transformers to capture global semantic sequences. However, both architectures encounter a significant bottleneck as they scale: over-smoothing. In deep GNNs, node representations tend to converge toward a uniform mean, losing the distinct features necessary for classification. Similarly, deep Transformers can suffer from a loss of local structural information. Consequently, most state-of-the-art ABSA models are restricted to "shallow" architectures (typically 2-3 layers), which limits their ability to model complex linguistic nuances.
The TextGT Architecture: A Double-View Approach
To address these limitations, the authors propose TextGT, a novel "double-view" graph Transformer. TextGT treats text as a dual-natured entity, alternating between a local, structure-constrained graph view and a global, semantic sequence view.
The core of the model is the TextGT block, which consists of three primary stages:
- TextGINConv (Graph View): This custom GNN layer processes the sentence as a graph derived from a dependency parser. It utilizes a dense implementation of Graph Isomorphism Network with Edge features (GINE), allowing the model to incorporate specific dependency labels (e.g., "nsubj", "nmod") directly into the word representations.
- Middle Layer: A normalization and ReLU activation stage that stabilizes the representations before transitioning views.
- Transformer Layer (Sequence View): A standard self-attention mechanism that treats the text as a global sequence. This allows every word to interact with every other word, regardless of their distance in the syntax tree.
The "Hinder" Effect: Depth as an Asset
The most significant insight of TextGT is the "hinder" effect created by alternating these views. By switching between a local graph convolution and a global self-attention mechanism, the two different smoothing processes essentially interfere with one another. This prevents the representations from collapsing into a uniform state, maintaining discriminative power even at greater depths.
While traditional ABSA models see a sharp decline in accuracy beyond three layers, TextGT demonstrates remarkable stability and even performance gains at depths of up to eight layers. This allows the model to learn significantly more complex linguistic patterns than its shallow predecessors.
Experimental Results and Performance
TextGT was evaluated on four standard ABSA benchmarks: Restaurant (SemEval 2014), Laptop (SemEval 2014), Rest16 (SemEval 2016), and a Twitter dataset. The model was tested using both BiLSTM and BERT backbones to demonstrate its versatility.
The results were compelling. TextGT+BERT achieved state-of-the-art performance, reaching 87.31% accuracy on the Restaurant dataset, outperforming established models like DualGCN and R-GAT. Beyond raw metrics, visualization of the model's attention maps revealed high interpretability. For example, in the phrase "This little place is wonderfully warm," the model correctly focused on the relationship between the aspect "place" and the sentiment-bearing "wonderfully warm," while effectively ignoring irrelevant modifiers like "little."
Conclusion and Future Directions
TextGT represents a significant step forward in ABSA by turning the "depth problem" into a strength. By tightly coupling syntactic and semantic views, it overcomes the over-smoothing barrier that has long hampered deep graph-based NLP. While the model currently relies on external dependency parsers—meaning parser errors can propagate—the underlying architecture provides a robust framework for future development. The authors intend to explore the application of the TextGT block to other complex NLP tasks and investigate more advanced methods for fusing convolutional and self-attention mechanisms.
Summary updated