Abstract
Extracting the relations that exist between two entities is an important step in numerous Web-related tasks such as information extraction. A supervised relation extraction system that is trained to extract a particular relation type might not accurately extract a new type of a relation for which it has not been trained. However, it is costly to create training data manually for every new relation type that one might want to extract. We propose a method to adapt an existing relation extraction system to extract new relation types with minimum supervision. Our proposed method comprises two stages: learning a lower-dimensional projection between different relations, and learning a relational classifier for the target relation type with instance sampling. We evaluate the proposed method using a dataset that contains 2000 instances for 20 different relation types. Our experimental results show that the proposed method achieves a statistically significant macro-average F-score of 62.77. Moreover, the proposed method outperforms numerous baselines and a previously proposed weakly-supervised relation extraction method.
Introduction
Information Extraction (IE) aims to identify semantic links between entities, such as recognizing that "Steve Jobs" and "Apple" are connected by a ceoOf relation. Traditional supervised learning approaches require thousands of manually labeled examples for each relation type, creating a significant bottleneck given the vast number of potential relations on the web. A model trained on financial news (e.g., "mergers and acquisitions") typically fails to generalize to other domains like cinema (e.g., "directors"). This paper addresses the Relation Adaptation problem: adapting a system trained on several "source" relations to a new "target" relation using only a minimal set of training instances.
Methodology
The proposed framework consists of two primary stages: Relational Mapping and One-sided Under-sampling.
Relational Mapping
The system extracts lexical and syntactic patterns from text and categorizes them into Relation-Specific (RS) patterns, which are unique to a single relation, and Relation-Independent (RI) patterns, which are shared across multiple relations. For example, "X directed Y" can apply to both corporate leadership and film production.
The method identifies RI patterns using Shannon entropy; patterns appearing across diverse entity pairs are treated as general "pivot" patterns. By constructing a bipartite graph connecting RS and RI patterns and applying spectral clustering, the system projects high-dimensional features into a lower-dimensional latent space. This allows similar relations to be grouped together even when they utilize different surface-level phrasing.
One-sided Under-sampling
To combat class imbalance—where source data vastly outweighs the few target examples—the authors introduced a custom under-sampling algorithm. This intelligently selects source data subsets most relevant to the target, preventing the target relation signal from being overwhelmed during training.
Experimental Setup
The method was evaluated using 20 relation types from the YAGO Ontology, with 100 entity pairs per relation. Web contexts were retrieved via the Yahoo BOSS API, yielding approximately 7,000 snippets per pair. To simulate a low-supervision environment, models were trained with 60 instances for source relations but only 10 instances for the target relation.
Results and Discussion
The experimental results demonstrate that the proposed method significantly outperforms existing approaches.
As shown in the performance table, the proposed "Comb.
- Samp." method achieved a macro-average F-score of 62.77, notably higher than the next best baseline (57.62) and the state-of-the-art cross-domain method by Jiang (2009) at 55.62. The method was particularly effective in precision (86.47), which is critical for maintaining the integrity of automated knowledge bases.
Key insights include:
- RI Patterns as Pivots: Shared linguistic "skeletons" allow for effective knowledge transfer between relations.
- Feature Selection: Using all available patterns can degrade performance; filtering for high-entropy, independent patterns creates a more robust bridge for transfer.
- Sampling Necessity: Without under-sampling, precision drops as the volume of source data drowns out the target examples.
Conclusion
This research presents a robust method for adapting relation extraction systems to novel types with minimal supervision. While entropy-based selection can occasionally introduce noise, the dual-stage approach of latent mapping and intelligent sampling provides a significant performance boost. Future work may explore scaling this method to thousands of relations and applying the adaptation logic to other classification tasks.
Summary updated