Discourse-level Relation Extraction via Graph Pooling

I-Hung Hsu, Xiao Guo, Prem Natarajan, Nanyun Peng

Abstract

The ability to capture complex linguistic structures and long-term dependencies among words in a passage is essential for discourse-level relation extraction (DRE) tasks. Graph Neural Networks (GNNs), which encode dependency graphs, have proven effective for DRE. However, relatively little attention has been paid to the receptive fields of GNNs, which are crucial for long texts requiring deep discourse understanding. In this work, we leverage a pooling-unpooling framework for DRE tasks. The pooling branch reduces graph size, enabling GNNs to obtain larger receptive fields with fewer layers, while the unpooling branch restores the graph to its original resolution for entity mention extraction. We propose Clause Matching (CM), a novel linguistically inspired graph pooling method. Experiments on two DRE datasets demonstrate that our models significantly improve over baselines when modeling long-term dependencies is required.

What Problem Does This Paper Address?

Most relation extraction (RE) models focus on single sentences. However, real-world information often spans entire paragraphs—a task known as Discourse-level Relation Extraction (DRE).

Example of discourse-level relation extraction where entities and their relations span multiple sentences.

While GNNs are excellent at encoding document structure via dependency trees, they suffer from a limited receptive field. To capture a relationship between words 20 steps apart, a standard GNN would need 20 layers, which often leads to "over-smoothing"—making node representations indistinguishable. This paper introduces a way for GNNs to "zoom out" to see global context without losing the fine-grained details needed to identify specific entities.

Methods

The authors propose a symmetric pooling-unpooling framework, drawing inspiration from the U-Net architecture used in computer vision.

The proposed pooling-unpooling framework for GNNs, featuring hierarchical graph reduction and restoration.

The Pooling-Unpooling Framework

  1. The Pooling Branch: This stage merges structurally similar or related nodes into "supernodes." This creates a compact hypergraph where a single GNN layer can cover a much larger portion of the original text.
  2. The Unpooling Branch: To extract specific relations, the model must return to the original word-level detail. The unpooling branch uses matching matrices from the pooling step to project information back from the compact hypergraph to the original resolution.

Clause Matching (CM)

A core contribution is Clause Matching (CM), a linguistically-driven graph pooling method. Unlike generic algorithms that merge nodes based only on connectivity, CM uses dependency logic. It identifies "core arguments" (like subjects and objects) and keeps them distinct, while merging "modifier" tokens (like adjectives or determiners) into their parent heads. This simplifies the graph while preserving the essential semantic structure of the sentences.

Datasets and Evaluation

The models were evaluated on two challenging DRE benchmarks:

  • CDR (Chemical-Disease Reactions): Identifying relations between chemicals and diseases in medical abstracts.
  • Cross-sentence n-ary: A biomedical dataset involving ternary (drug-gene-mutation) and binary relations that often span multiple sentences.

Key Insights and Results

  • Receptive Field is Critical: The authors demonstrate that the primary bottleneck for GNNs in long-form text is the inability to aggregate distant information efficiently.
  • Linguistic Inductive Bias: Generic graph pooling works, but pooling based on dependency types (CM) is more effective. It ensures that "supernodes" represent meaningful semantic units rather than random clusters.
  • Significant Performance Gains: On the CDR dataset, the pooling-unpooling models outperformed strong baselines like LSR and SSAN. Using SciBERT with CM pooling achieved an F1 score of 68.2, a notable jump over the standard GCN baseline (66.8).
  • Long-Distance Specialist: The framework showed the most dramatic improvements in cases where entities were furthest apart (8+ nodes away) or the input text was exceptionally long (375+ tokens).

Conclusion

This work shows that a pooling-unpooling framework effectively addresses the receptive field limitations of GNNs in discourse-level tasks. By incorporating linguistic structure through Clause Matching, the model achieves state-of-the-art results on complex biomedical extraction tasks. Future work will explore differentiable pooling and applying this framework to other document-level tasks like summarization.

Summary updated

Abstract

The ability to capture complex linguistic structures and long-term dependencies among words in the passage is essential for discourse-level relation extraction (DRE) tasks. Graph neural networks (GNNs), one of the methods to encode dependency graphs, have been shown effective in prior works for DRE. However, relatively little attention has been paid to receptive fields of GNNs, which can be crucial for cases with extremely long text that requires discourse understanding. In this work, we leverage the idea of graph pooling and propose to use pooling-unpooling framework on DRE tasks. The pooling branch reduces the graph size and enables the GNNs to obtain larger receptive fields within fewer layers; the unpooling branch restores the pooled graph to its original resolution so that representations for entity mention can be extracted. We propose Clause Matching (CM), a novel linguistically inspired graph pooling method for NLP tasks. Experiments on two DRE datasets demonstrate that our models significantly improve over baselines when modeling long-term dependencies is required, which shows the effectiveness of the pooling-unpooling framework and our CM pooling method.
BibTeX
@misc{hsu2021discourse,
  author = {I-Hung Hsu and Xiao Guo and Prem Natarajan and Nanyun Peng},
  title = {{Discourse-level Relation Extraction via Graph Pooling}},
  year = {2021},
  eprint = {2101.00124},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2101.00124},
}
Figure