Abstract
The ability to capture complex linguistic structures and long-term dependencies among words in a passage is essential for discourse-level relation extraction (DRE) tasks. Graph Neural Networks (GNNs), which encode dependency graphs, have proven effective for DRE. However, relatively little attention has been paid to the receptive fields of GNNs, which are crucial for long texts requiring deep discourse understanding. In this work, we leverage a pooling-unpooling framework for DRE tasks. The pooling branch reduces graph size, enabling GNNs to obtain larger receptive fields with fewer layers, while the unpooling branch restores the graph to its original resolution for entity mention extraction. We propose Clause Matching (CM), a novel linguistically inspired graph pooling method. Experiments on two DRE datasets demonstrate that our models significantly improve over baselines when modeling long-term dependencies is required.
What Problem Does This Paper Address?
Most relation extraction (RE) models focus on single sentences. However, real-world information often spans entire paragraphs—a task known as Discourse-level Relation Extraction (DRE).
While GNNs are excellent at encoding document structure via dependency trees, they suffer from a limited receptive field. To capture a relationship between words 20 steps apart, a standard GNN would need 20 layers, which often leads to "over-smoothing"—making node representations indistinguishable. This paper introduces a way for GNNs to "zoom out" to see global context without losing the fine-grained details needed to identify specific entities.
Methods
The authors propose a symmetric pooling-unpooling framework, drawing inspiration from the U-Net architecture used in computer vision.
The Pooling-Unpooling Framework
- The Pooling Branch: This stage merges structurally similar or related nodes into "supernodes." This creates a compact hypergraph where a single GNN layer can cover a much larger portion of the original text.
- The Unpooling Branch: To extract specific relations, the model must return to the original word-level detail. The unpooling branch uses matching matrices from the pooling step to project information back from the compact hypergraph to the original resolution.
Clause Matching (CM)
A core contribution is Clause Matching (CM), a linguistically-driven graph pooling method. Unlike generic algorithms that merge nodes based only on connectivity, CM uses dependency logic. It identifies "core arguments" (like subjects and objects) and keeps them distinct, while merging "modifier" tokens (like adjectives or determiners) into their parent heads. This simplifies the graph while preserving the essential semantic structure of the sentences.
Datasets and Evaluation
The models were evaluated on two challenging DRE benchmarks:
- CDR (Chemical-Disease Reactions): Identifying relations between chemicals and diseases in medical abstracts.
- Cross-sentence n-ary: A biomedical dataset involving ternary (drug-gene-mutation) and binary relations that often span multiple sentences.
Key Insights and Results
- Receptive Field is Critical: The authors demonstrate that the primary bottleneck for GNNs in long-form text is the inability to aggregate distant information efficiently.
- Linguistic Inductive Bias: Generic graph pooling works, but pooling based on dependency types (CM) is more effective. It ensures that "supernodes" represent meaningful semantic units rather than random clusters.
- Significant Performance Gains: On the CDR dataset, the pooling-unpooling models outperformed strong baselines like LSR and SSAN. Using SciBERT with CM pooling achieved an F1 score of 68.2, a notable jump over the standard GCN baseline (66.8).
- Long-Distance Specialist: The framework showed the most dramatic improvements in cases where entities were furthest apart (8+ nodes away) or the input text was exceptionally long (375+ tokens).
Conclusion
This work shows that a pooling-unpooling framework effectively addresses the receptive field limitations of GNNs in discourse-level tasks. By incorporating linguistic structure through Clause Matching, the model achieves state-of-the-art results on complex biomedical extraction tasks. Future work will explore differentiable pooling and applying this framework to other document-level tasks like summarization.
Summary updated