Abstract
This paper introduces the Learnable Syntax-Transport Graph Convolutional Network (LST-AGCN), a novel approach to Relation Extraction (RE) that treats linguistic dependency trees as weighted, learnable graphs. By employing a "soft pruning" strategy and an attention mechanism to aggregate information across multiple network depths, the model effectively filters syntactic noise while preserving structural integrity. LST-AGCN achieves state-of-the-art performance on the SemEval-2010 Task 8 benchmark, demonstrating the superiority of learnable syntax over rigid pruning methods.
Introduction
Relation Extraction (RE) is the task of identifying semantic relationships between entities in a sentence (e.g., recognizing a Component-Whole relationship). To perform this task, models often rely on dependency trees—grammatical diagrams showing how words modify one another. However, these trees are inherently noisy; not every word is relevant to a specific relation.
Previous attempts to mitigate this noise include "hard pruning" (cutting the tree to a shortest dependency path) and "fully connected graphs" (using attention between all words). Hard pruning often discards vital context, while fully connected graphs can lose the underlying syntactic structure entirely. The LST-AGCN seeks a middle ground: maintaining the tree structure while learning which branches are significant for a given relation.
Methodology: The LST-AGCN Architecture
The core innovation is the Syntax-Transport Graph, where connections between words are assigned weights between 0 and 1.
Learnable Weights and Soft Pruning
The model calculates weights using word embeddings, dependency types (e.g., "subject" vs. "modifier"), and contextual features from a BiLSTM. This "soft pruning" allows the model to dim the influence of irrelevant words without removing them entirely, preserving context that hard pruning might lose.
Multi-Layer Aggregation with Attention
Deep GCNs often suffer from "over-smoothing," where node representations become indistinguishable. To address this and the fact that different sentences require different reasoning depths, LST-AGCN runs for $K$ layers and uses an attention mechanism to aggregate outputs from every layer.
This architecture allows the model to "soft-choose" the optimal depth for each specific sentence, making it robust to varying sentence lengths and tree complexities.
Experimental Results
The model was evaluated on two major benchmarks: SemEval-2010 Task 8 and TACRED.
Performance on SemEval-2010
LST-AGCN achieved a state-of-the-art F1 score of 86.0 on the SemEval dataset, outperforming previous models like C-AGGCN and various LSTM-based baselines.
Key Insights
- Soft Pruning Effectiveness: Learning to weight edges is more effective than rigid, rule-based pruning. It allows the model to retain nodes that are semantically vital even if they are distant in the tree.
- Context-Dependent Syntax: The importance of a grammatical structure varies by context. For instance, the word "by" is critical for Cause-Effect relations but may be noise for Instrument-Agency relations.
- Ablation Studies: Removing learnable node importance ($A_D$) or edge transport ($A_T$) led to significant performance drops (to 84.7 and 84.9 respectively), confirming that both components are vital to the model's success.
Conclusion and Future Work
LST-AGCN demonstrates that treating syntax as a learnable transport graph significantly improves relation extraction. While the model currently relies on the initial quality of the dependency parser, future work could explore alternative parsing tools to mitigate "parsing noise." Additionally, investigating more sophisticated weight initialization strategies could further enhance the model's efficiency and accuracy.
Summary updated