Syn2DRE: Synergistic Dual Teacher-Student Framework for Document-level Relation Extraction

Khai Phan Tran, Xue LiarXiv 2026 · arXiv

TL;DR

Syn2DRE is a novel semi-supervised framework for Document-level Relation Extraction (DocRE) that overcomes the "confirmation bias" of traditional pseudo-labeling. By using a dual teacher-student architecture where two different model types (graph-based and transformer-based) cross-supervise each other, the system generates high-quality labels from unlabeled data in a single, efficient training run.

What Problem Does This Paper Address?

Extracting relationships between entities across an entire document (DocRE) is significantly harder than sentence-level extraction because it requires reasoning across multiple sentences. While deep learning models excel at this, they are "data-hungry," and manually annotating DocRE datasets is a nightmare—a single document in the popular DocRED dataset can require annotating roughly 38,000 candidate entity triples.

To bypass this, researchers use pseudo-labeling: training a model on a small labeled set, using it to label a large unlabeled set, and then retraining. However, this often leads to confirmation bias, where the model reinforces its own mistakes. Existing solutions to this bias are often computationally expensive, requiring multiple iterative rounds of "denoising" and retraining.

Methods

The authors introduce Syn2DRE (Synergistic Dual Teacher-Student), which uses two key strategies to produce reliable labels efficiently:

  1. Dual Teacher-Student Pairs: The framework maintains two separate student models ($s_1, s_2$) and two corresponding teacher models ($t_1, t_2$). The teachers are not trained via backpropagation; instead, they are updated using an Exponential Moving Average (EMA) of their respective students' weights. This creates a "stable" version of the model that provides more reliable targets.
  2. Unsupervised Synergistic Cross-supervision: To break the cycle of confirmation bias, the authors use two different architectures: KD-DocRE (transformer-based) and UGDRE (graph-based). During training, these models exchange "disagreeing" pseudo-labels. If Model A is highly confident about a relation where Model B is uncertain or disagrees, Model B learns from Model A’s prediction.
  3. Uncertainty-Aware Selection: Not all pseudo-labels are kept. The system uses Monte Carlo (MC) dropout to estimate uncertainty. If a teacher's predictions vary wildly across multiple forward passes, that label is discarded as unreliable.
Figure 1: An overview of the proposed Syn2DRE method.

Datasets

The framework was evaluated on five major benchmarks:

  • General Domain: DocRED and its refined version, Re-DocRED (English), and HacRED (Chinese).
  • Biomedical Domain: CDR (Chemical-Disease Relations) and GDA (Gene-Disease Associations).

Key Insights and Ideas

  • Architecture Diversity as a Filter: The most powerful idea here is that different model architectures (graph vs. transformer) have different "blind spots." By forcing them to teach each other specifically on instances where they disagree, the framework effectively filters out errors that a single model type would likely repeat.
  • Efficiency via Single-Pass Training: Unlike prior state-of-the-art methods that require multiple rounds of training and denoising, Syn2DRE achieves better results in a single end-to-end run.
  • Robustness to the "Long Tail": DocRE datasets are notoriously imbalanced, with a few common relations and many rare ones. Syn2DRE's ability to generate reliable synthetic labels significantly boosts performance on these infrequent "long-tail" relations.

Key Results

Syn2DRE achieved state-of-the-art (SOTA) results across all tested datasets:

  • DocRED/Re-DocRED: Consistently outperformed previous SOTA methods like UGDRE by +0.38 to +1.26 F1 points.
  • Biomedical Gains: The most dramatic improvement was on the CDR dataset, where Syn2DRE boosted the F1 score by 8.47 points over the previous best baseline.
Model performance on CDR and GDA datasets.
  • Long-tail Performance: On rare relations in Re-DocRED, it achieved an LT $F_1$ of 74.90, significantly higher than the 72.88 of the next best method (DURE).
Experimental results in long-tailed settings on Re-DocRED’s test set.
  • Computational Efficiency: Syn2DRE completed training in 26 hours, compared to 55 hours for UGDRE, while using only about 30% of the unlabeled data required by its predecessor.

Limitations and Future Work

Despite its success, the authors acknowledge a few hurdles:

  • Memory Overhead: Running two teacher-student pairs simultaneously nearly doubles the GPU memory requirement compared to single-model frameworks.
  • Labeled Data Dependency: While it uses unlabeled data effectively, it still requires a "non-trivial" amount of human-annotated data for the initial warm-up phase.
  • Fixed Thresholds: The current system uses fixed confidence thresholds for selecting pseudo-labels; the authors suggest that adaptive thresholds (learning when to trust the model) could be a promising future direction.

Summary updated

Abstract

Document-level Relation Extraction (DocRE) aims to identify relations between entities across a document but typically relies on costly large-scale annotations. While pseudo-labelling enables the use of low-quality data, existing methods often suffer from inefficiency and confirmation bias. To address these challenges, we propose Syn2DRE, a semi-supervised framework that combines self-supervised learning with cross-supervision in a lightweight teacher-student architecture to produce more reliable pseudo-labels. Additionally, we introduce a unified end-to-end training strategy to further improve efficiency of the framework. Experiments on multiple benchmarks show that Syn2DRE consistently outperforms prior state-of-the-art semi-supervised DocRE methods.
BibTeX
@misc{tran2026syn,
  author = {Khai Phan Tran and Xue Li},
  title = {{Syn2DRE: Synergistic Dual Teacher-Student Framework for Document-level Relation Extraction}},
  year = {2026},
  eprint = {2611.09342},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.09342},
}
Figure