Abstract
Relation Extraction (RE) is a fundamental task in information extraction, yet its application across diverse languages is hindered by the high cost of manual annotation. This paper presents Prompt-XRE, a novel framework designed for zero-shot Cross-Lingual Relation Extraction (XRE). By utilizing prompt-tuning on multilingual Sequence-to-Sequence (Seq2Seq) models like mBART, the framework bridges the gap between high-resource and low-resource languages. Our approach achieves state-of-the-art results on standard benchmarks and introduces a massive 0.9M-pair English-Chinese dataset to facilitate future research in the field.
Introduction
Relation Extraction (RE) involves identifying semantic connections between entities within a text—for instance, determining that "Steve Jobs" is the founder of "Apple Inc." While RE systems have reached high maturity in English, scaling these systems to the world's thousands of other languages remains a significant bottleneck.
The primary obstacle is data scarcity. Annotating relationships requires native linguistic expertise and is prohibitively expensive for most languages. Zero-Shot Cross-Lingual Relation Extraction (XRE) offers a solution by training models on high-resource source languages (e.g., English) and deploying them on target languages (e.g., Chinese or Arabic) without additional labeled data. However, traditional XRE methods often rely on complex features like dependency parsing, which are computationally intensive and prone to error propagation in low-resource settings.
Methodology: The Prompt-XRE Framework
Prompt-XRE addresses these challenges by reframing the RE task as a sequence generation problem. The authors leverage the mBART architecture, a multilingual pre-trained encoder-decoder model. Unlike standard fine-tuning, which can lead to "catastrophic forgetting" of the model's broad multilingual knowledge, prompt-tuning aligns the downstream task with the model's original pre-training objective (e.g., denoising or mask filling).
Designed Sequence-to-Sequence Prompts
The core innovation lies in the design of nine distinct prompt templates, categorized into three types:
- Hard Prompts: Manually crafted natural language strings that provide explicit instructions (e.g., "What is the type of relationship between Ent1 and Ent2?").
- Soft Prompts: These utilize artificial tokens or
[MASK]tokens to segment input. This allows the model to learn its own continuous representations of the task structure within its vector space. - Hybrid Prompts: A combination of natural language cues and soft tokens designed to provide both semantic guidance and structural flexibility.
Experimental Setup and Datasets
The researchers evaluated Prompt-XRE on two primary benchmarks:
- ACE05: A standard benchmark featuring English (En), Chinese (Zh), and Arabic (Ar). Despite its utility, it is relatively low-resource.
- WMT17-EnZh XRE: To address the need for larger datasets, the authors released a new corpus containing 0.9 million English-Chinese pairs. This was constructed from the WMT 2017 parallel corpus using automated tools (spaCy and PrePatt) to identify entities and their corresponding links across languages.
Key Insights and Results
Prompt-XRE demonstrated superior performance across all tested language pairs, establishing a new state-of-the-art (SOTA) for zero-shot XRE.
- Performance Gains: On the ACE05 English-to-Chinese transfer task, Prompt-XRE outperformed the previous SOTA model (GATE) by a significant 22.1 F1 points.
- Entity Emphasis: Prompts that explicitly highlighted entities (e.g., "The sentence of Sent includes Ent1 and Ent2") performed substantially better than those providing only the raw sentence. This suggests that guiding the model's attention to the relevant entities is critical for relation classification.
- The Efficacy of Soft Prompts: "Soft Prompt 3" emerged as one of the most effective templates. By avoiding the rigid syntax of natural language, soft prompts allow the model to better utilize the multilingual knowledge acquired during pre-training.
- Scalability: The model's success on the 0.9M-pair WMT dataset confirms that prompt-learning remains highly effective even as the volume of data increases.
Conclusion and Future Work
Prompt-XRE proves that prompt-tuning is a powerful mechanism for cross-lingual transfer, effectively "closing the gap" between pre-training and downstream tasks. While the results in English, Chinese, and Arabic are promising, future research should extend this framework to even lower-resource languages and explore automated prompt discovery methods to further optimize performance. Combining prompt-learning with adversarial training or data augmentation also remains a fertile ground for improving the robustness of zero-shot systems.
Summary updated