Abstract
Traditional event detection systems are often limited to narrow, predefined domains due to the high cost of human annotation. This paper presents a novel framework for open-domain event detection that leverages distant supervision from WordNet and Wikipedia. By automatically generating high-quality training data, the proposed system identifies a wide range of event types across diverse genres. Experimental results demonstrate that this approach outperforms traditional supervised models, achieving a 71.3% Strict F1 score on a new multi-domain benchmark.
Introduction: The Challenge of Open-Domain Detection
Most existing event detection research focuses on "closed-domain" scenarios, where models are trained on specific categories (e.g., "Conflict" or "Justice") within restricted genres like newswire. While effective in their niche, these models struggle with the "open-domain" reality of the web, where events can range from geological shifts to economic fluctuations.
The primary obstacle to scaling these systems is the "human-in-the-loop" bottleneck. Manually labeling every possible event type across all domains is prohibitively expensive. To address this, the authors explore whether "event-ness" can be learned as a general linguistic property through distant supervision, eliminating the need for exhaustive human labeling.
Methodology
The authors propose a two-phase pipeline: Training Data Generation and Event Detection.
Phase 1: Training Data Generation
The system uses WordNet to identify "eventualities"—actions, processes, and states. By mapping WordNet's taxonomic hierarchy to these categories, the authors automatically label verbs and nouns in the SemCor corpus.
To extend coverage to proper nouns and real-world events not found in standard dictionaries (e.g., specific historical events), the authors utilize Wikipedia. They train a specialized neural classifier, GC-BLSTM-Attn, on WordNet's textual definitions ("glosses"). This classifier learns to distinguish "eventive" definitions (e.g., "the act of paying money") from non-eventive ones (e.g., "meat from a domestic hog"). This classifier is then applied to Wikipedia article introductions to harvest new training examples.
Phase 2: Event Detection Model
The core detection model is a Bidirectional LSTM (BLSTM) equipped with a self-attention mechanism.
The model treats event detection as a sequence labeling task. To handle complex linguistic structures like multi-word expressions or discontinuous phrases (e.g., "turned the TV on"), the authors employ an extended tagging scheme (BIO + DB/DI). This allows the model to capture event "nuggets" that span multiple non-contiguous words.
Evaluation and Results
The system was evaluated against a new gold-standard benchmark, SW100, consisting of 100 articles from Simple English Wikipedia across 10 diverse domains (e.g., Architecture, Chemistry, Disaster, Economics).
Key results include:
- Superior Performance: The distantly supervised model (DS-BLSTM) achieved a 71.3% Strict F1 score, outperforming a standard supervised BLSTM (67.9%) trained on limited human-labeled data.
- Gloss Classification Accuracy: The attention-based gloss classifier reached 91.65% accuracy on WordNet and 85.00% on unseen Wikipedia concepts.
- Cross-Domain Robustness: The model maintained consistent performance across all 10 domains, suggesting that the learned features of "event-ness" are largely domain-independent.
Discussion: Insights and Limitations
A significant finding is the utility of dictionary "glosses" as a bridge between structured knowledge bases and unstructured text. The linguistic patterns used to define events are remarkably stable, allowing a model trained on definitions to generalize to real-world mentions. Furthermore, the study suggests that the sheer volume of distantly supervised data (11 times larger than typical human-labeled sets) compensates for the inherent noise in automatic labeling.
However, limitations remain. The model occasionally struggles with ambiguous nouns—for instance, distinguishing "payment" as an action from "payment" as a physical sum of money. Additionally, highly technical or rare domain-specific terms may be missed if they lack sufficient representation in the underlying knowledge sources.
Conclusion
This work demonstrates that distant supervision is a viable and powerful alternative to manual annotation for open-domain event detection. By treating "event-ness" as a universal linguistic property and leveraging the vast knowledge in WordNet and Wikipedia, the authors provide a scalable path forward for understanding events across the entire web.
Summary updated