Abstract
Low-resource relation extraction (LRE) aims to extract relations from limited labeled corpora. Existing work takes advantages of self-training or distant supervision to expand the limited labeled data in the data-driven approaches, while the selection bias of pseudo labels may cause the error accumulation in subsequent relation classification. To address this issue, this paper proposes fmLRE, an iterative feedback method based on feature mapping similarity calculation to improve the accuracy of pseudo labels. First, it calculates the similarities between pseudo-label and real-label data of the same category in a feature mapping space based on semantic features of labeled dataset after feature projection. Then, it fine-tunes the initial model according to the iterative process of reinforcement learning. Finally, the similarity is used as a threshold for screening high-precision pseudo-labels and the basis for setting different rewards, which also acts as a penalty term for the loss function of the relation classifier. Experimental results demonstrate that fmLRE achieves state-of-the-art performance compared with strong baselines on two public datasets.
Introduction
Relation Extraction (RE) is the task of identifying semantic relationships between entities in text, such as recognizing that "Steve Jobs" and "Apple" share a "founder of" relationship. While modern deep learning models excel at this when trained on massive datasets, they struggle in low-resource scenarios where labeled data is scarce.
A common solution is "self-training," where a model labels unlabeled data (creating "pseudo-labels") to expand its own training set. However, this often leads to selection bias: if the initial model makes a mistake, it trains on its own errors, leading to a cycle of error accumulation. The fmLRE framework was developed to break this cycle by creating a more robust way to validate and utilize these pseudo-labels.
Methodology
The fmLRE framework is an iterative feedback method that uses two primary "engines" to improve label quality:
Feature Mapping Similarity (Metric Learning)
Instead of just trusting the model's classification output, fmLRE projects sentences into a specialized "feature mapping space." It calculates the similarity between a new pseudo-labeled sample (the "query") and the average feature of known samples in that category (the "support set"). If the similarity is high, the label is trusted. This category-aware approach is more sensitive to relational nuances than global similarity measures.
Reinforcement Learning (RL) Feedback
The similarity score is used as a reward signal in an RL framework. High similarity provides positive feedback, while low similarity acts as a penalty term in the loss function. This forces the model to align its internal representations more closely with real-world label categories.
Noise Filtering and Model Backbone
The model includes a "strike system" for unlabeled data. If a sample repeatedly fails to meet the similarity threshold over several iterations, it is flagged as noise and removed. Additionally, replacing a standard BERT backbone with Matching the Blanks (MTB)—a model pre-trained specifically for relational goals—significantly raises the performance ceiling.
Experimental Results
The model was evaluated on the SemEval 2010 Task 8 and TACRED datasets. fmLRE consistently outperformed strong baselines like GradLRE and MetaSRE across different ratios of labeled data.
On SemEval, with only 5% labeled data, fmLRE achieved an 82.11% F1 score, outperforming the next best baseline by 2.5%. On the challenging TACRED dataset, in a very sparse 3% labeled data scenario, fmLRE reached 50.83% F1, a significant jump over GradLRE’s 47.37%.
Discussion and Conclusion
The core innovation of fmLRE is using similarity in a mapped space for three distinct purposes: as a gatekeeper (filtering labels), a reward (for RL), and a penalty (for the classifier loss). While highly effective, the authors acknowledge limitations such as the "zero-shot gap"—the inability to identify relations never seen during training—and the current need for manual tuning of the similarity threshold. Future work will explore adaptive learning to automate threshold selection and improve generalization for entirely novel relations.
Summary updated