Introduction
Open Information Extraction (Open IE) systems are proficient at harvesting massive amounts of factual triples from unstructured web text, typically in the form of (subject, relation string, object). However, these relation strings (e.g., "is the father of") are often ambiguous and lack the formal structure required for downstream applications like question answering or semantic search. To bridge this gap, relation integration aims to map these free-text relations to standardized predicates within a structured Knowledge Graph (KG) like Wikidata or Freebase.
The primary challenge in this task is ambiguity. A text relation like "parent" could correspond to several KG relations, such as father or mother, depending on the gender of the subject. Traditional "local" models treat each triple independently, which can lead to global inconsistencies—for instance, predicting that an individual has two biological fathers because the model failed to consider the broader relational context.
The CoRI Framework
To address these limitations, the authors introduce CoRI (Collective Relation Integration), a two-stage framework designed to move beyond isolated predictions toward a holistic, graph-based understanding.
Two-Stage Inference
- Candidate Generation (Local Stage): An initial model generates a set of candidate KG relations for each Open IE triple. These predictions are made independently, focusing on the local context of the specific triple.
- Collective Inference (Global Stage): This stage treats the initial guesses from the first stage as input features. It examines an entity alongside all its neighbors' predicted relations simultaneously. By looking at the "big picture," the model can identify and correct logical errors.
Stacked Generalization
To ensure the second stage learns to handle errors effectively, the authors employ stacked generalization. The training data is partitioned into folds; the first-stage model is trained on one set of folds to generate realistic (and potentially erroneous) predictions for the remaining folds. This prevents the second-stage model from simply memorizing perfect labels and forces it to learn the structural constraints of the KG.
Data Augmentation (DA)
A significant bottleneck in training relation integration models is the scarcity of parallel data—instances where both a text extraction and a corresponding KG fact exist for the same entity pair. CoRI overcomes this by leveraging the "unmatched" portions of the target KG.
The authors "hallucinate" training data by taking entities present in the KG but absent from the text corpus and synthesizing pseudo-extractions. By using co-occurrence statistics between text relations and KG predicates, they create fake text relations for existing KG facts. To ensure these pseudo-extractions are useful, they use BM25 to select data that closely mirrors the distribution of the real-world text data, preventing domain shift.
Experimental Evaluation
The researchers evaluated CoRI on two benchmarks: ReVerb + Freebase and ReVerb + Wikidata. They utilized a challenging 20/80 split, where only 20% of the available data was used for training, to test the model's robustness in low-resource settings.
Key Results
CoRI consistently outperformed state-of-the-art local models, such as OpenKI, across multiple metrics. On the Freebase dataset, the Area Under the Curve (AUC) improved significantly from 0.677 to 0.748. Similarly, on Wikidata, the AUC rose from 0.716 to 0.780.
The precision-recall curves demonstrate that CoRI maintains higher precision at higher recall levels compared to baseline methods. The addition of Data Augmentation (CoRI + DA) further boosts performance, highlighting the value of leveraging the structural knowledge inherent in the unmatched KG. A qualitative analysis supports these findings; for example, CoRI corrected a local error where 'Iowa' was predicted to be contained by 'Mahaska County' by recognizing the broader geographical hierarchy.
Conclusion and Future Work
CoRI demonstrates that knowledge integration is fundamentally a collective task. By treating relations as part of a web rather than isolated facts, the model achieves superior accuracy and logical consistency. However, the approach is not without limitations; its effectiveness depends on the density of the source graph. Future research may explore more advanced graph-based architectures, such as Graph Neural Networks (GNNs), to further refine the global inference process.
Summary updated