Abstract
Open Information Extraction (OpenIE) structures information from natural language text in the form of (subject, predicate, object) triples. Supervised OpenIE is, in principle, only possible for English, for which plenty of labeled data exists. Recent research efforts tackled multilingual OpenIE by means of zero-shot transfer from English, with massively multilingual language models as vehicles of transfer. Given that OpenIE is a highly syntactic task, such transfer tends to fail for languages that are syntactically more complex and distant from English. In this work, we propose two Linguistic Feature Projection strategies to alleviate the situation, having observed the failure of transferring from English to German, Arabic, and Japanese. The strategies, namely (i) reordering of words in source-language utterances to match the target language word order and (ii) code-switching, lead to training data that contains features of both the source (English) and target language. Experiments render both strategies effective and mutually complementary on German, Arabic, and Japanese. Additionally, we propose a third strategy tailored for English-Japanese transfer by (iii) inserting Japanese case markers into English utterances, which leads to further performance gains.
Introduction
Open Information Extraction (OpenIE) is the task of automatically pulling structured information—specifically (subject, predicate, object) triples—out of plain text. While English-language OpenIE has benefited from large-scale supervised datasets like OpenIE4, most other languages lack such resources.
The standard approach to multilingual OpenIE is zero-shot cross-lingual transfer, where a model (such as mBERT or XLM-R) is trained on English data and applied to target languages. However, OpenIE is a highly syntactic task, meaning it depends heavily on word order and sentence structure. For languages with word orders significantly different from English (SVO), such as Japanese (SOV) or Arabic (VSO), zero-shot transfer often fails completely. Initial experiments showed that a standard mBERT model trained on English achieved a 0% F1 score when tested on Japanese, highlighting a critical "syntactic gap."
Methodology: Linguistic Feature Projection (LFP)
To bridge this gap, we propose Linguistic Feature Projection (LFP). Instead of forcing the model to jump directly from English to a foreign language, we create an intermediate "pseudo-English" training set that adopts the target language's syntactic and lexical "flavor." This allows the model to learn on structures that look and feel like the target language, even though the core vocabulary remains largely English.
We employ three primary strategies:
- Reordering (RO): Using a word aligner, we shuffle the words in English training sentences to match the word order of the target language. For example, "Ivan will give a book to Anna" becomes "Ivan book Anna to give will."
- Code-Switching (CS): We randomly replace English words with their translated counterparts in the target language to help the model learn lexical alignments.
- Case Marker Insertion (CM): Specifically for Japanese, we insert Japanese "case markers" (functional particles like wa, wo, and ni) into the English sentences to signal grammatical roles.
Experimental Results
We evaluated our approach using BenchIE, a comprehensive multi-faceted benchmark, and introduced Japanese BenchIE, the first gold-standard OpenIE benchmark for Japanese, created by translating and manually annotating the original English BenchIE.
Our results demonstrate that LFP significantly improves performance across diverse languages, outperforming standard zero-shot transfer and traditional annotation projection:
- Japanese: Performance jumped from 0.0% F1 to 10.61% F1.
- Arabic: Improved from 2.64% F1 to 13.45% F1.
- German: Improved from 5.89% F1 to 11.45% F1.
The study confirms that for OpenIE, the "syntactic gap" (word order) is a much bigger hurdle than the "lexical gap" (vocabulary). Reordering (RO) provided much larger gains than just swapping words via code-switching (CS), especially for languages most distant from English.
Conclusion
Linguistic Feature Projection acts as a bridge, allowing multilingual models to "land halfway" between source and target languages. By fine-tuning on data that incorporates features of both languages, the final transfer becomes much smoother. While LFP significantly improves results for syntactically distant languages, challenges remain regarding discontinuous spans and the reliance on upstream machine translation quality. Future work will explore these strategies against Large Language Models (LLMs) like GPT-4.
Summary updated