Multilingual SubEvent Relation Extraction: A Novel Dataset and Structure Induction Method

Viet Dac Lai, Hieu Man, Linh Ngo Van, Franck Dernoncourt, Thien Huu Nguyen

Introduction

Subevent Relation Extraction (SRE) is a critical task in Natural Language Processing that involves identifying hierarchical relationships between events—for instance, determining that a "bombing" is a constituent part of a larger "terrorist attack." Accurate SRE requires a deep understanding of both spatial and temporal context. However, the field has historically struggled with two primary issues: "context blindness" caused by over-reliance on fixed dependency paths, and a heavy bias toward English-language datasets.

This paper introduces OT-SRE, a structure induction method that leverages Optimal Transport to dynamically construct context-rich graphs. Furthermore, the authors present mSubEvent, the first high-quality multilingual dataset for subevent extraction, covering five typologically diverse languages.

The Challenge of Context Blindness

Traditional SRE models often rely on the shortest dependency path between two event mentions. While useful, this approach frequently ignores vital context words that fall outside the grammatical path. For example, in the sentence "They implemented the proposal... developed frameworks...", the word "proposal" is essential for linking "implemented" and "developed," yet it is often excluded by standard dependency-path methods.

Methodology: OT-SRE

To overcome these limitations, the authors propose a "Structure Induction" approach. Instead of a static graph, OT-SRE learns to build a custom graph for every document:

  1. Optimal Transport (OT) Alignment: The model treats words on the dependency path as "anchors." It uses Optimal Transport—a mathematical framework for optimizing mass movement between distributions—to align "non-path" words with these anchors based on syntactic proximity and semantic similarity.
  2. Pruning with the "NIL" Filter: To minimize noise, the model includes a "NIL" data point. Words that are most efficiently transported to NIL are deemed irrelevant and pruned, ensuring the resulting graph is both dense in information and low in noise.
  3. Graph Convolutional Networks (GCN): The induced, pruned graph is then processed by a GCN to generate final representations for relation classification.

The mSubEvent Dataset

A major contribution of this work is the mSubEvent dataset. It contains over 46,000 event mentions and 3,900 subevent relations across five languages: English, Danish, Spanish, Turkish, and Urdu. Sourced from Wikipedia categories like aviation accidents and natural disasters, it provides a robust benchmark for cross-lingual SRE research.

Distribution of event mentions across the five languages in the mSubEvent dataset.

Key Results

OT-SRE demonstrates superior performance across multiple benchmarks. On the English HiEve dataset, it achieved a state-of-the-art (SOTA) F1 score of 68.9, a 3% improvement over previous methods. More importantly, the model maintained consistent performance across the diverse languages in the mSubEvent dataset.

Performance metrics (F1 scores) for event and relation extraction across languages.

As shown in the table above, while the model performs exceptionally well in English (0.96 Relation score), there is a noticeable performance gap in languages like Turkish (0.66 Relation score). This highlights the ongoing challenge of "language-agnostic" reasoning and the importance of the mSubEvent dataset in identifying these disparities.

Conclusion and Future Work

The OT-SRE method proves that moving beyond fixed dependency paths via Optimal Transport significantly enhances subevent extraction. While the introduction of mSubEvent marks a milestone for multilingual NLP, the results indicate that cross-lingual transfer remains a difficult task. Future research may extend this OT-based structure induction to other complex relations, such as event causality.

Summary updated

BibTeX
@inproceedings{lai2022multilingual,
  author = {Viet Dac Lai and Hieu Man and Linh Ngo Van and Franck Dernoncourt and Thien Huu Nguyen},
  title = {{Multilingual SubEvent Relation Extraction: A Novel Dataset and Structure Induction Method}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2022},
  doi = {10.18653/v1/2022.findings-emnlp.407},
}
Figure