TL;DR
This paper introduces a new task: multilingual event linking to Wikidata, which involves connecting mentions of events in various languages to their corresponding entries in a knowledge base. They've created a large dataset for this task and show that advanced neural models significantly outperform traditional methods, though crosslingual linking remains a significant challenge.
What Problem Does This Paper Address?
Information extraction systems often struggle to understand and connect events described across different languages. While entity linking (connecting names to knowledge base entries) is well-studied, event linking – which requires considering participants, temporal, and spatial attributes – is less explored, especially in a multilingual context. Existing methods often focus on English, leaving a gap in understanding events globally. This paper aims to bridge this gap by creating a multilingual event linking task and dataset, enabling models to ground event mentions from diverse languages to a unified knowledge base (Wikidata).
Methods
The authors adapt the "retrieve+rank" pipeline, popularized by the BLINK system for entity linking, for event linking. This involves two stages:
- Retriever (Bi-encoder): A bi-encoder model (using mBERT or XLM-RoBERTa) encodes both the mention's context and candidate event descriptions. It then scores potential event matches using a dot product of their embeddings, retrieving the top-k candidates.
- Ranker (Cross-encoder): A cross-encoder model takes the mention context and a candidate event description as input and outputs a final score to re-rank the top-k candidates from the retriever, identifying the most likely event.
They also use BM25, a traditional TF-IDF based retrieval method, as a baseline.
Datasets
- Newly Introduced Dataset: The core contribution is a large-scale, multilingual dataset for event linking. It comprises 1.8 million mentions across 44 languages that refer to over 10,900 events from Wikidata. This dataset was compiled by leveraging hyperlinks within Wikipedia articles and connecting them to Wikidata.
- Wikinews Evaluation Set: To test out-of-domain generalization, a smaller evaluation set was created using Wikinews articles, providing a more realistic scenario for real-world applications.
Key Insights and Ideas
- Wikidata as an Interlingua: Using Wikidata as the knowledge base is crucial because it acts as a common ground (interlingua) that connects mentions from numerous languages.
- Crosslingual is Harder: The crosslingual task (linking mentions to English event descriptions) is consistently more challenging than the multilingual task (linking to descriptions in the same language as the mention). This suggests that language-specific nuances in event descriptions are highly informative.
- Neural Models Outperform Baselines: Advanced bi-encoder and cross-encoder models significantly outperform the traditional BM25 baseline, demonstrating the power of deep learning for this task.
- Temporal Reasoning is Key: The dataset's structure, with many event sequences, highlights the importance of temporal reasoning for accurate event linking, an area where current systems sometimes struggle.
- Diverse Mention Types: Event mentions can be more varied than entity mentions, including temporal or spatial expressions, adding complexity to the linking task.
Key Results
- Accuracy Gains: Bi-encoder and cross-encoder models achieved significantly higher accuracy than the BM25 baseline on both multilingual and crosslingual tasks. For instance, on the multilingual task, mBERT-cross achieved ~90% accuracy, while BM25+ was around 50%.
- Crosslingual Performance: While still outperforming BM25, crosslingual performance was generally lower than multilingual performance across all models. For example, mBERT-cross achieved ~76% accuracy on the crosslingual task compared to ~90% on the multilingual task.
- Wikinews Generalization: The models showed good cross-domain performance on the Wikinews set but moderate zero-shot transfer, indicating that unseen domains and events remain a challenge. Including article dates and titles in the context improved performance by 4-12%.
Limitations and Future Work
- Dataset Scope: The current dataset primarily focuses on "eventive nouns" and might not fully cover verbal events.
- Out-of-Domain Challenges: The Wikinews evaluation highlighted that generalizing to new domains and unseen events is difficult.
- Temporal/Spatial Reasoning: Systems still struggle with temporal and spatial expressions, and distinguishing between events in sequences.
- Future Directions:
- Expanding the event dictionary to include hierarchical event structures.
- Increasing mention coverage to include more verbal events.
- Improving temporal reasoning and handling of temporal/spatial expressions.
- Incorporating events that lack English descriptions.
- Investigating performance differences between planned (e.g., sports) and unplanned (e.g., wars) events.
Summary updated