MrGCN: Mirror Graph Convolution Network for Relation Extraction with Long-Term Dependencies

Xiao Guo, I-Hung Hsu, Wael AbdAlmageed, Premkumar Natarajan, Nanyun Peng

Abstract

The ability to capture complex linguistic structures and long-term dependencies among words in the passage is essential for discourse-level relation extraction (DRE) tasks. Graph neural networks (GNNs), one of the methods to encode dependency graphs, have been shown effective in prior works for DRE. However, relatively little attention has been paid to receptive fields of GNNs, which can be crucial for cases with extremely long text that requires discourse understanding. In this work, we leverage the idea of graph pooling and propose to use a pooling-unpooling framework on DRE tasks. The pooling branch reduces the graph size and enables the GNNs to obtain larger receptive fields within fewer layers; the unpooling branch restores the pooled graph to its original resolution so that representations for entity mentions can be extracted. We propose Clause Matching (CM), a novel linguistically inspired graph pooling method for NLP tasks. Experiments on two DRE datasets demonstrate that our models significantly improve over baselines when modeling long-term dependencies is required, which shows the effectiveness of the pooling-unpooling framework and our CM pooling method.

Introduction

Relation Extraction (RE) aims to identify semantic connections between entities in text. While sentence-level RE is relatively mature, Discourse-level Relation Extraction (DRE) presents a significant challenge: entities of interest often appear in different sentences, requiring the model to understand long-range dependencies across an entire document.

Example of DRE where entities like 'adriamycin' and 'cystitis' are separated by several sentences.

Graph Neural Networks (GNNs) operating on dependency graphs are a popular approach for DRE. However, they suffer from a limited "receptive field." To capture a dependency between nodes 10 hops apart, a standard GNN requires 10 layers. Stacking GNNs to this depth often leads to "over-smoothing," where node representations become indistinguishable, hindering the model's ability to differentiate between distinct entities and their roles.

Methodology: The Mirror Graph Convolution Network (MrGCN)

To address the receptive field limitation, we propose the Mirror Graph Convolution Network (MrGCN), which utilizes a pooling-unpooling framework. This architecture allows the model to aggregate global information efficiently without the need for excessive depth.

Pooling-Unpooling Framework

The MrGCN architecture consists of two symmetric branches:

  1. The Pooling Branch: This branch iteratively reduces the graph's size by merging nodes into "supernodes." By creating increasingly compact hypergraphs, a single GNN layer can cover a much larger distance in the original text.
  2. The Unpooling Branch: Once high-level features are extracted from the coarse graphs, the unpooling branch restores the graph to its original resolution. This ensures that the final representations are mapped back to specific entity mentions, which is necessary for the final relation classification.
The MrGCN architecture showing the hierarchical pooling and unpooling process to expand the receptive field.

Clause Matching (CM) Pooling

A key innovation of this work is Clause Matching (CM) pooling, a linguistically-driven method for graph reduction. Unlike structural pooling methods that ignore semantics, CM identifies "core arguments" (e.g., subjects, objects) and "non-core" modifiers (e.g., adjectives, determiners). It merges modifiers into their respective head words while preserving the essential clause structure. This "linguistic pruning" simplifies the graph while maintaining the critical "who-did-what" logic necessary for relation extraction.

Experimental Results

We evaluated MrGCN on two major DRE benchmarks: the Chemical-Disease Reactions (CDR) dataset and a Cross-sentence n-ary biomedical dataset.

Performance Gains

On the CDR dataset, MrGCN with CM pooling achieved an F1 score of 68.2 using SciBERT, a significant improvement over the standard GCN baseline (66.8). Notably, the framework provided substantial boosts even when using static embeddings like GloVe, demonstrating that the structural improvements are independent of the underlying language model.

Long-Distance Dependencies

The effectiveness of the pooling-unpooling framework is most evident when analyzing performance relative to the distance between entities. MrGCN showed the most dramatic gains in cases where entities were separated by more than 8 hops in the dependency graph. This confirms that the hierarchical pooling successfully expands the receptive field to capture long-term dependencies that standard GNNs miss.

Conclusion

MrGCN introduces an efficient way to handle long-range dependencies in DRE by adopting a pooling-unpooling architecture. By incorporating linguistic knowledge through Clause Matching pooling, the model effectively prunes irrelevant information while preserving core semantic structures. Future work will explore learnable pooling methods and the application of this framework to other document-level NLP tasks such as coreference resolution.

Summary updated

Abstract

The ability to capture complex linguistic structures and long-term dependencies among words in the passage is essential for discourse-level relation extraction (DRE) tasks. Graph neural networks (GNNs), one of the methods to encode dependency graphs, have been shown effective in prior works for DRE. However, relatively little attention has been paid to receptive fields of GNNs, which can be crucial for cases with extremely long text that requires discourse understanding. In this work, we leverage the idea of graph pooling and propose to use pooling-unpooling framework on DRE tasks. The pooling branch reduces the graph size and enables the GNNs to obtain larger receptive fields within fewer layers; the unpooling branch restores the pooled graph to its original resolution so that representations for entity mention can be extracted. We propose Clause Matching (CM), a novel linguistically inspired graph pooling method for NLP tasks. Experiments on two DRE datasets demonstrate that our models significantly improve over baselines when modeling long-term dependencies is required, which shows the effectiveness of the pooling-unpooling framework and our CM pooling method.
BibTeX
@misc{guo2021mrgcn,
  author = {Xiao Guo and I-Hung Hsu and Wael AbdAlmageed and Premkumar Natarajan and Nanyun Peng},
  title = {{MrGCN: Mirror Graph Convolution Network for Relation Extraction with Long-Term Dependencies}},
  year = {2021},
  eprint = {2101.00124},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2101.00124},
}
Figure