Abstract
Dependency trees convey rich structural information that is proven useful for extracting relations among entities in text. However, how to effectively make use of relevant information while ignoring irrelevant information from the dependency trees remains a challenging research question. Existing approaches employing rule based hard-pruning strategies for selecting relevant partial dependency structures may not always yield optimal results. In this work, we propose Attention Guided Graph Convolutional Networks (AGGCNs), a novel model which directly takes full dependency trees as inputs. Our model can be understood as a soft-pruning approach that automatically learns how to selectively attend to the relevant sub-structures useful for the relation extraction task. Extensive results on various tasks including cross-sentence n-ary relation extraction and large-scale sentence-level relation extraction show that our model is able to better leverage the structural information of the full dependency trees, giving significantly better results than previous approaches.
Introduction
Relation Extraction (RE) is the task of identifying semantic relationships between entities in text (e.g., determining that a specific drug "treats" a specific disease). To do this effectively, models often rely on dependency trees, which map the grammatical structure of a sentence.
However, dependency trees are inherently noisy. Traditionally, researchers used "hard pruning"—manually defined rules to strip away parts of the tree, such as keeping only the shortest path between entities. The fatal flaw of hard pruning is that it can accidentally discard "trigger words" that are essential for identifying the relation but fall outside the pruned structure. Attention Guided Graph Convolutional Networks (AGGCNs) solve this by letting the model learn which parts of the full tree to ignore and which to emphasize through a "soft pruning" mechanism.
Methodology
The AGGCN architecture transforms the dependency tree into a fully connected, edge-weighted graph. It consists of three main components:
- Attention Guided Layer (Soft Pruning): Instead of deleting edges, the model uses a multi-head self-attention mechanism to assign weights to every possible edge between nodes. This allows the model to "softly" prune the tree by assigning near-zero weights to irrelevant connections while highlighting crucial ones, even if they are many hops away in the original tree.
- Densely Connected Layer: To capture complex, long-distance dependencies, the authors incorporate dense connections (inspired by DenseNet), where each sub-layer receives the concatenated outputs of all preceding sub-layers. This facilitates better information flow and allows for deeper, more effective GCNs.
- Linear Combination Layer: This layer integrates the representations from the $N$ different attention heads into a unified hidden representation for the next block.
Experimental Results
The model was evaluated on several prominent benchmarks, including PubMed (cross-sentence n-ary relation extraction), TACRED (large-scale sentence-level RE), and SemEval-10 Task 8.
AGGCN established new state-of-the-art (SOTA) performance across the board:
- n-ary Relation Extraction: On the PubMed dataset, the model outperformed the previous best (GS GLSTM) by 6.8% in ternary relation accuracy.
- TACRED: The "C-AGGCN" variant achieved an F1 score of 69.0, significantly outperforming the previous SOTA C-GCN (66.4).
- SemEval-10: Achieved an F1 score of 85.7, surpassing previous dependency-based and sequence-based models.
Analysis showed that while other models' performance degraded as sentences got longer and dependency trees became more complex, AGGCN remained stable, proving its ability to filter noise effectively.
Conclusion
AGGCNs represent a significant advancement in relation extraction by replacing rigid, rule-based dependency tree pruning with a learnable self-attention mechanism. By transforming dependency trees into fully connected graphs and utilizing dense connections, the model effectively handles the noise inherent in full dependency trees. While the $O(n^2)$ complexity of fully connected graphs remains a consideration for extremely long documents, the model's ability to retain critical context makes it a powerful tool for complex extraction tasks.
Summary updated