Abstract
Crosslingual Relation and Event Extraction (REE) often suffers from "monolingual bias," where models overfit to source-language patterns and fail to generalize to target languages. This paper presents CCCAR (Class- and Category-based Crosslingual Alignment of Representations), a framework that mitigates this bias by aligning class-specific prototypes and universal word categories (e.g., parts-of-speech) across languages. By filtering out distracting sentence-specific context and employing an adaptive alignment coefficient, CCCAR achieves state-of-the-art results on the ACE 2005 benchmark for English, Chinese, and Arabic.
Introduction
Relation and Event Extraction (REE) are fundamental tasks in Information Extraction, identifying semantic relationships between entities (e.g., Employee-of) and occurrences of specific events (e.g., Attack). While performance is high in high-resource languages like English, the cost of annotating data for every language is prohibitive. Crosslingual transfer learning aims to bridge this gap by training on a source language and deploying on a target language using multilingual embeddings like mBERT.
However, standard transfer methods often encounter monolingual bias, where the model over-indexes on source-specific syntax. Furthermore, adversarial methods that align feature distributions often ignore class labels, leading to class misalignment—for instance, incorrectly aligning an "Attack" event in Chinese with a "Meeting" event in English due to structural similarities.
Methodology: The CCCAR Framework
The proposed CCCAR framework addresses these challenges through three primary alignment strategies and a noise-filtering mechanism.
1. Class-aware Alignment
To prevent class misalignment, CCCAR learns "prototype" vectors for each class (e.g., a specific event type). For the source language, these are derived from gold labels. For the unlabeled target language, the model generates weighted average vectors based on its current predictions. The training objective minimizes the distance between source and target prototypes of the same class, ensuring semantic consistency across the language barrier.
2. Word Category-based Alignment
Syntax often exhibits more universality than vocabulary. CCCAR aligns representations for Universal Parts-of-Speech (UPOS) and Dependency Relations. An embedding table serves as a shared "anchor" to which both languages must map their respective representations.
3. Context Information Filtering
A challenge with contextualized embeddings (like mBERT) is that they conflate category information with specific sentence context. To isolate "pure" word category features, the authors utilize a Gradient Reversal Layer (GRL). This forces the model to learn representations that are sufficient for category identification but insufficient for reconstructing the original context words.
4. Adaptive Coefficient
To handle noisy initial predictions in the target language, the alignment loss is introduced gradually. The weight of the alignment pressure increases as the model's confidence in its target-language predictions grows over time.
Experimental Results
The framework was evaluated on the ACE 2005 dataset across English (EN), Chinese (ZH), and Arabic (AR), covering six transfer directions.
CCCAR consistently outperformed previous state-of-the-art models and standard adversarial baselines. Key findings include:
- Event Detection (ED): The model achieved a significant average improvement of 7.7% in F1 score over the baseline, as shown in the table above.
- Relation and Argument Extraction: Performance in Relation Extraction (RE) and Event Argument Extraction (EAE) improved by approximately 1.9% to 2.0% on average.
- Data Efficiency: CCCAR demonstrated high efficiency, matching baseline performance while using only 60% to 80% of the source language's labeled data.
Conclusion
CCCAR demonstrates that aligning universal semantic (class) and syntactic (category) anchors while filtering out language-specific context significantly enhances crosslingual REE. Future work may extend these alignment strategies to other complex tasks like coreference resolution and investigate ways to reduce dependency on external pre-processing toolkits.
Summary updated