Abstract
Knowledge Graphs (KGs) are essential for modern information retrieval but suffer from a "data hunger" problem when dealing with emerging, niche relations. Traditional Relation Extraction (RE) models require extensive labeled data, which is often unavailable for new relations. We present Neural Snowball, a framework that combines the iterative nature of bootstrapping with deep Relational Siamese Networks (RSN). By learning a universal metric of relational similarity from existing data, Neural Snowball can autonomously "grow" a training set from unlabeled text starting from just a few seed examples. Our experiments show that this approach nearly doubles the performance of standard fine-tuning in 5-shot scenarios.
Introduction
The rapid growth of information necessitates the constant addition of new relations to Knowledge Graphs. However, the long-tail nature of these relations means that for many, only a handful of labeled instances exist. This "few-shot" scenario is a significant challenge for supervised learning. While bootstrapping techniques—which use seeds to find more examples—have been used in the past, they often rely on rigid, hand-crafted patterns that fail to capture the linguistic diversity of natural language.
Neural Snowball addresses this by transferring "similarity logic" rather than just features. It uses prior knowledge from well-documented relations to identify when two sentences express the same underlying relationship, even if that relationship is entirely new to the model.
Methodology: The Snowball Loop
The core of the framework is an iterative process driven by two neural components: the Relational Siamese Network (RSN) and a set of binary relation classifiers.
Relational Siamese Networks (RSN)
The RSN is pre-trained on large-scale datasets (like FewRel) to learn a distance metric. Instead of classifying a sentence into a specific category, it learns to determine if two sentences express the same relation. This allows the RSN to act as a high-precision gatekeeper during the data collection process.
Iterative Expansion
The "snowball" effect is achieved through two alternating phases:
- Entity Expansion: Starting with seed sentences, the model identifies other sentences in an unlabeled corpus (e.g., Wikipedia) that contain the same entity pairs. The RSN filters these candidates to ensure they actually express the target relation.
- Relation Expansion: A binary classifier is trained on the current set of validated instances. This classifier then searches for new entity pairs that likely share the same relation. The RSN again validates these new discoveries before they are added to the training pool.
Experimental Results
We evaluated Neural Snowball using the FewRel benchmark and a Wikipedia unlabeled corpus containing approximately 900,000 instances. We tested the framework using both CNN and BERT architectures to ensure generalizability.
In a 5-shot setup (only 5 seed examples), Neural Snowball with a BERT encoder achieved an F1-score of 47.26. This is a dramatic improvement over simple fine-tuning (25.10) and traditional bootstrapping methods like BREDS (17.58). As the number of seeds increased to 15, the F1-score climbed to 72.06, demonstrating the framework's ability to effectively leverage even small amounts of additional supervision.
Discussion and Limitations
A key insight from our study is the "comfort zone" effect. While the RSN is excellent at maintaining high precision (often over 82% for top-5 selections), it tends to favor linguistic patterns that are semantically similar to the initial seeds. This can limit the "recall" or diversity of the discovered instances. Future work will focus on developing adaptive RSNs that can update their similarity metrics during the snowball process to better explore diverse linguistic expressions.
Conclusion
Neural Snowball represents a significant step forward for open-ended relation learning. By combining the strengths of deep metric learning with iterative bootstrapping, it provides a scalable and accurate way to populate Knowledge Graphs with niche relations, reducing the reliance on massive human-annotated datasets.
Summary updated