Introduction
Event detection in natural language processing involves identifying "who did what to whom." In Chinese news corpora, this task is particularly complex, requiring the integration of three sub-tasks: Word Sense Disambiguation (WSD), Semantic Role Labeling (SRL), and Event Detection (ED). PKU_HIT is a modular system designed to address these challenges by leveraging automated instance expansion to mitigate data scarcity and combining constituent and dependency-based syntactic features to improve role labeling.
Methodology
The PKU_HIT system employs a three-stage pipeline:
- Word Sense Disambiguation (WSD): A Naïve Bayesian classifier is used to determine the specific meaning of target verbs. To address limited labeled data, the authors implemented Instance Expansion based on a "one sense per 3-gram" heuristic, automatically extracting thousands of new training examples from a large untagged Chinese news corpus.
- Semantic Role Labeling (SRL): This module follows a four-step process: parsing, pruning, identification, and classification. The innovation lies in a rich feature set that integrates 34 constituent-based features (hierarchical structure) and 35 dependency-based features (functional relationships), marking a significant attempt to fuse these syntactic views for Chinese SRL.
- Event Detection (ED): A rule-based module that populates situation formulas (derived from WSD) with arguments identified by the SRL module.
Experimental Setup
The system was evaluated using the SemEval-2010 Task 11 dataset (4,608 sentences, 100 target predicates). For instance expansion, a monolingual corpus of the 2001 People’s Daily (approx. 116MB) provided 2,145 additional sense-labeled instances. The Chinese sense-tagged corpus (STC) served as a reference for validating the 3-gram heuristic.
Results and Discussion
Word Sense Disambiguation
The WSD module achieved a top Macro Accuracy of 83.81%, ranking 3rd among 7 participants. Analysis revealed that local context is paramount; systems using a small window size (±3 words) significantly outperformed those with larger windows (±15). Interestingly, automated data expansion slightly decreased performance, likely due to noise introduced by the simple 3-gram heuristic and the fine granularity of word senses in the task.
Semantic Role Labeling and Event Detection
The SRL module reached an F-Score of 55.71%. A critical finding was the "pruning bottleneck": the system struggled more with identifying argument boundaries (AB) than with labeling the roles themselves. As shown in the performance breakdown, if boundaries are correctly identified, the accuracy for semantic roles (SR) and event roles (ER) remains relatively high.
The overall ED accuracy was lower (approximately 20.19%), illustrating how errors in WSD and SRL propagate and compound through the pipeline.
Conclusion
PKU_HIT demonstrates the potential of combining diverse syntactic features for event detection but also highlights significant challenges. The current pruning algorithm, which relies on "gold-standard" logic applied to noisy automatic parses, remains a major bottleneck. Future work will focus on developing more robust pruning methods and refining mapping rules between different parsing standards to improve overall system performance.
Summary updated