Semi-Supervised Learning for Relation Extraction

Guodong Zhou, Junhui Li, Longhua Qian, Qiaoming ZhuIJCNLP 2008 · OpenReview

Abstract

Relation Extraction (RE) is a fundamental task in Natural Language Processing (NLP) that involves identifying semantic relationships between entities in text. While supervised learning models achieve high accuracy, they are constrained by the need for large-scale manually labeled datasets. This paper introduces a hybrid semi-supervised approach that combines the boundary-defining capabilities of Support Vector Machines (SVMs) with the structural modeling of Label Propagation (LP). By using weighted support vectors from a bootstrapped SVM as initial seeds for graph-based propagation, the proposed method effectively captures the manifold structure of unlabeled data. Evaluations on the ACE RDC 2003 corpus show that this approach significantly outperforms standard SVM and LP baselines, particularly when labeled data is scarce.

Introduction

The primary challenge in Relation Extraction is the "labeling bottleneck." Supervised methods are accurate but expensive to scale, while unsupervised methods often lack the precision required for specific relation types. Semi-supervised methods attempt to bridge this gap, yet they often face trade-offs: bootstrapping techniques can introduce noise, and standard Label Propagation is computationally intensive and sensitive to initial labeling.

Zhou et al. propose a "best of both worlds" solution. Their method is designed to be computationally efficient, capture the natural clustering (manifold structure) of unlabeled data, and focus on the most informative examples by using Support Vectors (SVs) as the foundation for label propagation.

Methodology

The proposed framework operates in two distinct phases:

1. SVM Bootstrapping with Co-training

Starting with a small labeled set, the model employs a modified BootProject algorithm. It uses Random Feature Projection to create multiple "views" of the data, a co-training strategy that enhances robustness. The model iteratively classifies unlabeled data, incorporating the most confident predictions into the training set. Crucially, the output of this phase is a set of weighted Support Vectors (SVs). These SVs represent the critical instances that define the decision boundaries between different relation types.

2. Label Propagation (LP) via Support Vectors

Rather than constructing a graph from the entire labeled dataset, the authors use the bootstrapped SVs as the primary sources of label information. In this graph-based model, instances are vertices, and edges are weighted by similarity. Labels "flow" from the weighted SVs through the graph to unlabeled instances until the system reaches a stable state. By using SVs as "anchors," the model ensures that propagation begins from the most mathematically significant points in the feature space.

Experimental Evaluation

The researchers evaluated the model using the ACE RDC 2003 corpus, focusing on 24 subtypes of relations. To further test the utility of unlabeled data, they incorporated the ACE RDC 2004 corpus as an additional source of unlabeled instances.

Results and Analysis

The experimental results demonstrate the efficiency and accuracy of the hybrid approach. As shown in the performance comparison below, the method excels even with minimal labeled data.

Comparison of F-measure performance across different labeling ratios and methods.

Key Findings:

  • Superior Low-Resource Performance: With only 5% of the labeled data, the proposed method (LP via weighted SVs) achieved an F-measure of 46.5, significantly outperforming a standard SVM (35.4) and standard Label Propagation (43.1).
  • The Importance of Weighting: Using weighted support vectors (based on SVM alpha values) consistently outperformed unweighted versions. This indicates that the geometric importance of a boundary case in the SVM directly correlates to its influence in the graph-based model.
  • Computational Efficiency: By focusing on support vectors—which typically comprise only 15–20% of the data—the model reduces the complexity of the graph-based LP phase without sacrificing accuracy.
  • Manifold Smoothing: Label Propagation acts as a smoother, allowing the model to account for natural clusters in the unlabeled data that a standard SVM might overlook.

Conclusion

This hybrid approach effectively addresses the labeling bottleneck in Relation Extraction by combining the strengths of SVMs and Label Propagation. Future work may explore more sophisticated similarity metrics for graph construction and the application of this method to more diverse domains, such as medical or legal text processing. Additionally, automating hyperparameter tuning for the co-training process could further enhance the model's usability in real-world applications.

Summary updated

BibTeX
@inproceedings{zhou2008semi,
  author = {Guodong Zhou and Junhui Li and Longhua Qian and Qiaoming Zhu},
  title = {{Semi-Supervised Learning for Relation Extraction}},
  booktitle = {IJCNLP},
  year = {2008},
  url = {https://openreview.net/forum?id=ytBPUMJ7DH},
}
Figure