Cross-lingual Structure Transfer for Relation and Event Extraction

Ananya Subburathinam, Di Lu, Heng Ji, Jonathan May, Shih-Fu Chang, Avirup Sil, Clare R. Voss

Introduction

Information Extraction (IE) is a critical component in the construction of large-scale knowledge graphs, involving the identification of relations (e.g., Person X works for Organization Y) and events (e.g., Person Z attended a Meeting). While state-of-the-art IE systems have achieved impressive results, they are typically data-intensive, requiring extensive human-annotated corpora. This poses a significant barrier for "under-resourced" languages where manual annotation is prohibitively expensive and time-consuming.

This paper introduces a novel framework that addresses this challenge by transferring structural knowledge from high-resource languages, such as English, to low-resource target languages. The core idea is to map sentences into a shared "multilingual common semantic space" using Graph Convolutional Networks (GCNs) and universal syntactic features, enabling effective extraction without the need for large-scale target-language training data.

Methodology: Building a Common Semantic Space

The proposed pipeline creates a language-agnostic representation through four key stages:

  1. Universal Dependency Parsing: Sentences are converted into standardized tree structures using the Universal Dependencies framework, focusing on functional relationships (e.g., subject, verb, object) rather than surface word order.
  2. Language-Universal Representations: Each node in the dependency tree is represented by a vector combining multilingual word embeddings (e.g., fastText), universal Part-of-Speech (POS) tags, dependency roles, and entity type labels.
  3. GCN Encoder: A Graph Convolutional Network (GCN) processes these trees. Unlike sequence-based models, GCNs operate directly on the dependency structure, capturing long-range context that remains consistent across languages despite variations in word order.
  4. Task-Specific Classifiers: The contextualized representations are passed to softmax layers for predicting relation types or event argument roles.
The proposed architecture for cross-lingual structure transfer, showing the GCN encoder and common semantic space.

Experimental Setup

The framework was evaluated using the ACE 2005 (Automatic Content Extraction) corpus, which includes gold-standard annotations for English (8,738 mentions), Chinese (9,317 mentions), and Arabic (4,731 mentions). Preprocessing involved UDPipe for universal dependency parsing and Stanford CoreNLP for basic text processing.

Key Insights

The success of this approach rests on several key observations:

  • Structural Invariance: While surface-level syntax varies, the underlying dependency structure of events is remarkably similar across languages. Training on trees allows the model to learn language-agnostic patterns.
  • Feature Complementarity: Symbolic features (POS tags, dependency paths) provide a stable skeleton for transfer, while distributional features (word embeddings) provide necessary semantic nuance.
  • Robustness of GCNs: GCNs are more effective than LSTMs for cross-lingual transfer because they are inherently robust to the structural shifts that occur during translation or mapping between languages.

Results and Performance

The cross-lingual transfer approach demonstrated remarkable efficiency. In Event Argument Role Labeling, the model transferred from English to Chinese achieved a 59.0% F1 score, nearly matching the 59.3% F1 of a supervised model trained directly on Chinese data.

Performance of cross-lingual transfer for event argument role labeling, illustrating the 3,000-mention milestone.

A significant finding is the "3,000 Mention Milestone": the transfer approach consistently matched the performance of supervised models trained on approximately 3,000 manually annotated mentions in the target language. Furthermore, ablation studies highlighted that dependency relation embeddings were the most impactful feature, significantly boosting F1 scores in Chinese Relation Extraction.

Limitations and Future Directions

While effective, the system is sensitive to the quality of the initial dependency parser; errors in low-resource language parsing can propagate through the network. Additionally, performance is slightly lower when using automatically extracted, noisy entity mentions compared to "perfect" gold-standard mentions.

Future research aims to incorporate visual features (e.g., images or videos from news sources) and external knowledge bases to further ground the common semantic space and reduce the reliance on purely textual features.

Summary updated

BibTeX
@inproceedings{subburathinam2019cross,
  author = {Ananya Subburathinam and Di Lu and Heng Ji and Jonathan May and Shih-Fu Chang and Avirup Sil and Clare R. Voss},
  title = {{Cross-lingual Structure Transfer for Relation and Event Extraction}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2019},
  doi = {10.18653/v1/d19-1030},
}
Figure