PKU_HIT: An Event Detection System Based on Instances Expansion and Rich Syntactic Features

Shiqi Li, Pengyuan Liu, Tiejun Zhao, Qin Lu, Hanjing LiSemEval @ ACL 2010

Introduction

Event detection in natural language processing involves identifying "who did what to whom." In Chinese news corpora, this task is particularly complex, requiring the integration of three sub-tasks: Word Sense Disambiguation (WSD), Semantic Role Labeling (SRL), and Event Detection (ED). PKU_HIT is a modular system designed to address these challenges by leveraging automated instance expansion to mitigate data scarcity and combining constituent and dependency-based syntactic features to improve role labeling.

Methodology

The PKU_HIT system employs a three-stage pipeline:

  1. Word Sense Disambiguation (WSD): A Naïve Bayesian classifier is used to determine the specific meaning of target verbs. To address limited labeled data, the authors implemented Instance Expansion based on a "one sense per 3-gram" heuristic, automatically extracting thousands of new training examples from a large untagged Chinese news corpus.
  2. Semantic Role Labeling (SRL): This module follows a four-step process: parsing, pruning, identification, and classification. The innovation lies in a rich feature set that integrates 34 constituent-based features (hierarchical structure) and 35 dependency-based features (functional relationships), marking a significant attempt to fuse these syntactic views for Chinese SRL.
  3. Event Detection (ED): A rule-based module that populates situation formulas (derived from WSD) with arguments identified by the SRL module.

Experimental Setup

The system was evaluated using the SemEval-2010 Task 11 dataset (4,608 sentences, 100 target predicates). For instance expansion, a monolingual corpus of the 2001 People’s Daily (approx. 116MB) provided 2,145 additional sense-labeled instances. The Chinese sense-tagged corpus (STC) served as a reference for validating the 3-gram heuristic.

Results and Discussion

Word Sense Disambiguation

The WSD module achieved a top Macro Accuracy of 83.81%, ranking 3rd among 7 participants. Analysis revealed that local context is paramount; systems using a small window size (±3 words) significantly outperformed those with larger windows (±15). Interestingly, automated data expansion slightly decreased performance, likely due to noise introduced by the simple 3-gram heuristic and the fine granularity of word senses in the task.

WSD performance across different system configurations.

Semantic Role Labeling and Event Detection

The SRL module reached an F-Score of 55.71%. A critical finding was the "pruning bottleneck": the system struggled more with identifying argument boundaries (AB) than with labeling the roles themselves. As shown in the performance breakdown, if boundaries are correctly identified, the accuracy for semantic roles (SR) and event roles (ER) remains relatively high.

Performance metrics for Semantic Role Labeling across different tag types.

The overall ED accuracy was lower (approximately 20.19%), illustrating how errors in WSD and SRL propagate and compound through the pipeline.

Conclusion

PKU_HIT demonstrates the potential of combining diverse syntactic features for event detection but also highlights significant challenges. The current pruning algorithm, which relies on "gold-standard" logic applied to noisy automatic parses, remains a major bottleneck. Future work will focus on developing more robust pruning methods and refining mapping rules between different parsing standards to improve overall system performance.

Summary updated

BibTeX
@inproceedings{li2010pku,
  author = {Shiqi Li and Pengyuan Liu and Tiejun Zhao and Qin Lu and Hanjing Li},
  title = {{PKU\_HIT: An Event Detection System Based on Instances Expansion and Rich Syntactic Features}},
  booktitle = {SemEval @ ACL},
  year = {2010},
}
Figure