Abstract
Open domain relation extraction systems identify relation and argument phrases in a sentence without relying on any underlying schema. However, current state-of-the-art relation extraction systems are available only for English because of their heavy reliance on linguistic tools such as part-of-speech taggers and dependency parsers. We present a cross-lingual annotation projection method for language independent relation extraction. We evaluate our method on a manually annotated test set and present results on three typologically different languages. We release these manual annotations and extracted relations in 61 languages from Wikipedia.
Introduction
Open Relation Extraction (RE) is a critical task in information extraction that aims to identify arbitrary phrases representing relationships between arguments in text. Unlike closed RE, which operates on a fixed schema, Open RE is more flexible but significantly more dependent on high-quality linguistic preprocessing, such as dependency parsing and part-of-speech tagging. Consequently, most advanced Open RE systems are restricted to English, leaving a vast majority of the world's languages underserved. This research addresses this bottleneck by introducing a cross-lingual projection method that leverages existing English-centric tools for multilingual extraction.
Methodology: Cross-lingual Projection
The proposed pipeline bypasses the need for native linguistic tools in the target language by utilizing Machine Translation (MT) as a bridge. The process consists of three primary stages:
- Translation and Alignment: A source sentence is translated into English using a standard MT system. This step also generates word-to-word alignments between the source and translated text.
- English Extraction: An established English Open RE system, OLLIE, is applied to the translated sentence to extract triples in the form
(arg1; relation; arg2). - Projection: The extracted English phrases are mapped back to the source language. To avoid the "word salad" effect of naive alignment, a novel algorithm selects the most similar contiguous phrase in the source language based on BLEU scores.
Experimental Setup and Data
The system was applied to the entire Wikipedia corpus for 61 languages, creating a massive multilingual knowledge resource. For evaluation, professional linguists manually annotated 1,000 relations for three typologically diverse languages: French, Hindi, and Russian. These manual annotations serve as a gold standard to measure the validity and accuracy of the projected relations.
Results and Discussion
The performance of the projection method was evaluated using two metrics: the percentage of extractions judged "valid" by humans and the BLEU score similarity between automatically projected phrases and human-selected "gold" phrases.
As shown in the results, French achieved the highest validity at 81.6%. Interestingly, while Russian had a lower validity rate (63.5%), it achieved the highest BLEU score (0.62). This discrepancy is attributed to the length of the relation phrases; Russian relations were significantly shorter (averaging 1.8 words) compared to French (3.6) or Hindi (4.1), making them easier to project accurately. The study highlights a clear trade-off: longer, more complex relations in languages like Hindi are harder to project coherently, leading to lower similarity scores.
Conclusion
This work demonstrates that cross-lingual projection is a viable strategy for scaling Open RE to low-resource languages without requiring native dependency parsers. By leveraging the maturity of MT systems, the researchers successfully extracted millions of relations across 61 languages. Future work involves using these projected "silver standard" relations to train native Open RE models, potentially eliminating the translation step in the future.
Summary updated