Introduction
Relation Extraction (RE) is a fundamental NLP task focused on identifying semantic relationships between entities in text. While supervised learning has seen great success, the reliance on large-scale labeled datasets remains a bottleneck. Semi-supervised Relation Extraction (SS-RE) attempts to leverage small labeled sets alongside vast unlabeled corpora.
However, traditional "cooperation-based" methods—such as self-training or dual-module systems—often suffer from semantic drift. These models tend to reach a consensus prematurely, focusing on "easy" unlabeled sentences that closely resemble the labeled data. This leads to insufficient exploration of the diverse linguistic expressions of a relation. To address this, the authors propose AdvSRE, a framework that shifts the paradigm from cooperation to competition.
The AdvSRE Framework
AdvSRE introduces a challenging minimax game on unlabeled data to force the model to explore the entire semantic space.
1. The Minimax Game
At the heart of AdvSRE is a competition between a Generator ($G$) and a Discriminator ($D$). The Generator aims to produce "fraudulent" sentence representations that mimic real relations. Conversely, the Discriminator must learn to distinguish these fakes from real unlabeled sentences while simultaneously classifying real sentences into their correct relation categories. This adversarial pressure prevents the model from becoming overconfident on a narrow subset of data.
2. Bayesian Generative Process (BGP)
To avoid "mode collapse"—a common issue where generators produce repetitive, low-quality samples—AdvSRE employs a Bayesian approach. Using Stochastic Gradient Hamiltonian Monte Carlo (SGHMC), the system samples parameters for the generator, ensuring the generated "fake" data is diverse and covers the entire latent semantic space.
3. Relational Contrastive Features (RCF)
The Discriminator utilizes contrastive learning to maximize the utility of limited labeled data. It learns to "pull" representations of sentences with the same relation closer together while "pushing" different relations apart in the latent space.
Experimental Evaluation
AdvSRE was evaluated on two major benchmarks: SemEval-2010 Task 8 and TACRED. TACRED is particularly challenging due to its size (42 relations) and highly skewed distribution, where nearly 79% of instances belong to the "no_relation" category.
Key Results
AdvSRE achieved new state-of-the-art (SOTA) performance across various labeled data proportions:
- TACRED: With only 15% labeled data, AdvSRE achieved an F1 score of 62.53, outperforming the previous SOTA (GradLRE) by 2.6%.
- SemEval: The model showed an average improvement of 1.71% F1 over GradLRE.
- Superior Recall: AdvSRE demonstrated exceptional recall (reaching 70.52 early in training on TACRED), indicating a superior ability to generalize and identify relations in unlabeled data compared to fully-supervised baselines.
Visualizing Exploration vs. Exploitation
The core advantage of AdvSRE lies in its ability to "explore" rather than just "exploit." While cooperative models produce dense, narrow clusters, AdvSRE's adversarial training leads to more scattered and diverse representations.
As shown in the t-SNE visualizations, AdvSRE (e) produces more distinct and spread-out clusters compared to methods like MetaSRE or GradLRE. This confirms that the minimax game successfully forces the model to learn a broader, more robust definition of each relation, effectively capturing the "long tail" of linguistic variety.
Conclusion
AdvSRE demonstrates that competition can be more effective than cooperation in semi-supervised settings. By challenging the discriminator with a diverse Bayesian generator, the model avoids the pitfalls of semantic drift and achieves superior generalization. This competitive minimax framework offers a promising direction for other NLP tasks where labeled data is scarce.
Summary updated