Generic and Trend-aware Curriculum Learning for Relation Extraction in Graph Neural Networks

Nidhi Vakil, Hadi Amiri

Abstract

We present a generic and trend-aware curriculum learning approach for graph neural networks. It extends existing approaches by incorporating sample-level loss trends to better discriminate easier from harder samples and schedule them for training. The model effectively integrates textual and structural information for relation extraction in text graphs. Experimental results show that the model provides robust estimations of sample difficulty and shows sizable improvement over the state-of-the-art approaches across several datasets.

Introduction

Relation Extraction (RE) is the fundamental task of identifying semantic connections between entities, such as determining if a specific gene is causal for a particular phenotype. While Graph Neural Networks (GNNs) have become a standard tool for this task, they often suffer from two primary limitations: information loss and training inefficiency. As information propagates through multiple layers (hops) in a GNN, the nuanced textual details of individual nodes can become "washed out." Furthermore, standard training regimes treat all samples equally, ignoring the inherent difficulty differences between simple and complex graph topologies.

To address these challenges, we propose a framework that combines a text-integrated Graph Text Neural Network (GTNN) with a novel "trend-aware" curriculum learning strategy, Trend-SL.

Methodology

Graph Text Neural Network (GTNN)

The GTNN architecture is designed to mitigate the loss of local textual features during graph convolution. It employs an encoder-decoder structure where initial text embeddings (e.g., Doc2Vec) are not only used for graph propagation but are also fed directly into the final decoder layer. This "shortcut" ensures that the model retains the specific textual identity of entities while simultaneously leveraging broader structural information from the graph.

The GTNN architecture integrates a Graph Neural Encoder with a Textual Summary Encoder to preserve entity-specific information during the decoding process.

Trend-aware SuperLoss (Trend-SL)

The core algorithmic contribution is Trend-SL, a curriculum learning method that dynamically weights samples based on their learning trajectory. Unlike traditional methods that rely on a static snapshot of current loss, Trend-SL introduces a trend indicator ($\Delta$). This indicator tracks the loss trajectory of a specific sample over the last $k$ iterations.

Trend-aware SuperLoss (Trend-SL) distinguishes between easy and hard instances by analyzing the trajectory of the loss function over time.

The logic behind Trend-SL is intuitive: if a sample has a high error but that error is dropping rapidly, the curriculum identifies it as "becoming easy" and increases its training weight. Conversely, if the error is stagnant or rising, the sample is classified as "hard" or noisy and is deprioritized to prevent the model from overfitting on ambiguous data.

Experimental Evaluation

We evaluated our model on three distinct datasets: GDPR (Gene, Disease, Phenotype Relation), PGR (Gene Phenotype Relation), and a Wikipedia-based link prediction dataset. The GDPR dataset was specifically curated to test "differential diagnosis" capabilities, requiring the model to distinguish between diseases with highly similar clinical descriptions.

Key Results

  • Performance Gains: GTNN achieved an average improvement of 8.6 F1 points over standard GNN baselines that lack direct text embedding shortcuts.
  • Curriculum Impact: The addition of the Trend-SL curriculum provided a further 0.7 F1 point boost, resulting in an average F1 of 89.9 across all datasets.
  • Stability: Trend-SL significantly reduced "inversions"—instances where a sample's difficulty status fluctuates wildly—leading to a more stable and robust training process.

Conclusion

By integrating direct textual access with a trend-aware curriculum, our framework effectively handles the complexities of relation extraction in large-scale graphs. The ability to distinguish between samples that are genuinely hard and those that are simply in the process of being learned allows for more efficient and accurate model training. Future work will explore more sophisticated time-series fitting techniques to further refine trend estimation and expand the application of trend-aware curricula to other graph-based tasks.

Summary updated

Abstract

We present a generic and trend-aware curriculum learning approach for graph neural networks. It extends existing approaches by incorporating sample-level loss trends to better discriminate easier from harder samples and schedule them for training. The model effectively integrates textual and structural information for relation extraction in text graphs. Experimental results show that the model provides robust estimations of sample difficulty and shows sizable improvement over the state-of-the-art approaches across several datasets.
BibTeX
@inproceedings{vakil2022generic,
  author = {Nidhi Vakil and Hadi Amiri},
  title = {{Generic and Trend-aware Curriculum Learning for Relation Extraction in Graph Neural Networks}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2022},
  url = {https://openreview.net/forum?id=VThi57EKId},
}
Figure