Relation Extraction with Convolutional Network over Learnable Syntax-Transport Graph

Kai Sun, Richong Zhang, Yongyi Mao, Samuel Mensah, Xudong Liu

Abstract

This paper introduces the Learnable Syntax-Transport Graph Convolutional Network (LST-AGCN), a novel approach to Relation Extraction (RE) that treats linguistic dependency trees as weighted, learnable graphs. By employing a "soft pruning" strategy and an attention mechanism to aggregate information across multiple network depths, the model effectively filters syntactic noise while preserving structural integrity. LST-AGCN achieves state-of-the-art performance on the SemEval-2010 Task 8 benchmark, demonstrating the superiority of learnable syntax over rigid pruning methods.

Introduction

Relation Extraction (RE) is the task of identifying semantic relationships between entities in a sentence (e.g., recognizing a Component-Whole relationship). To perform this task, models often rely on dependency trees—grammatical diagrams showing how words modify one another. However, these trees are inherently noisy; not every word is relevant to a specific relation.

Previous attempts to mitigate this noise include "hard pruning" (cutting the tree to a shortest dependency path) and "fully connected graphs" (using attention between all words). Hard pruning often discards vital context, while fully connected graphs can lose the underlying syntactic structure entirely. The LST-AGCN seeks a middle ground: maintaining the tree structure while learning which branches are significant for a given relation.

Methodology: The LST-AGCN Architecture

The core innovation is the Syntax-Transport Graph, where connections between words are assigned weights between 0 and 1.

Learnable Weights and Soft Pruning

The model calculates weights using word embeddings, dependency types (e.g., "subject" vs. "modifier"), and contextual features from a BiLSTM. This "soft pruning" allows the model to dim the influence of irrelevant words without removing them entirely, preserving context that hard pruning might lose.

Multi-Layer Aggregation with Attention

Deep GCNs often suffer from "over-smoothing," where node representations become indistinguishable. To address this and the fact that different sentences require different reasoning depths, LST-AGCN runs for $K$ layers and uses an attention mechanism to aggregate outputs from every layer.

The LST-AGCN architecture featuring multi-layer GCNs with attention-based aggregation.

This architecture allows the model to "soft-choose" the optimal depth for each specific sentence, making it robust to varying sentence lengths and tree complexities.

Experimental Results

The model was evaluated on two major benchmarks: SemEval-2010 Task 8 and TACRED.

Performance on SemEval-2010

LST-AGCN achieved a state-of-the-art F1 score of 86.0 on the SemEval dataset, outperforming previous models like C-AGGCN and various LSTM-based baselines.

Comparison of F1 scores on the SemEval-2010 Task 8 dataset, showing LST-AGCN achieving state-of-the-art results.

Key Insights

  • Soft Pruning Effectiveness: Learning to weight edges is more effective than rigid, rule-based pruning. It allows the model to retain nodes that are semantically vital even if they are distant in the tree.
  • Context-Dependent Syntax: The importance of a grammatical structure varies by context. For instance, the word "by" is critical for Cause-Effect relations but may be noise for Instrument-Agency relations.
  • Ablation Studies: Removing learnable node importance ($A_D$) or edge transport ($A_T$) led to significant performance drops (to 84.7 and 84.9 respectively), confirming that both components are vital to the model's success.

Conclusion and Future Work

LST-AGCN demonstrates that treating syntax as a learnable transport graph significantly improves relation extraction. While the model currently relies on the initial quality of the dependency parser, future work could explore alternative parsing tools to mitigate "parsing noise." Additionally, investigating more sophisticated weight initialization strategies could further enhance the model's efficiency and accuracy.

Summary updated

Abstract

A large majority of approaches have been proposed to leverage the dependency tree in the relation classification task. Recent works have focused on pruning irrelevant information from the dependency tree. The state-of-the-art Attention Guided Graph Convolutional Networks (AGGCNs) transforms the dependency tree into a weighted-graph to distinguish the relevance of nodes and edges for relation classification. However, in their approach, the graph is fully connected, which destroys the structure information of the original dependency tree. How to effectively make use of relevant information while ignoring irrelevant information from the dependency trees remains a challenge in the relation classification task. In this work, we learn to transform the dependency tree into a weighted graph by considering the syntax dependencies of the connected nodes and persisting the structure of the original dependency tree. We refer to this graph as a syntax-transport graph. We further propose a learnable syntax-transport attention graph convolutional network (LST-AGCN) which operates on the syntax-transport graph directly to distill the final representation which is sufficient for classification. Experiments on Semeval-2010 Task 8 and Tacred show our approach outperforms previous methods.
BibTeX
@inproceedings{sun2020relation,
  author = {Kai Sun and Richong Zhang and Yongyi Mao and Samuel Mensah and Xudong Liu},
  title = {{Relation Extraction with Convolutional Network over Learnable Syntax-Transport Graph}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.1609/aaai.v34i05.6423},
}
Figure