Abstract
This paper proposes a method for extracting events from newspapers using a hierarchical statistical approach. Candidate sentences are first selected via trigger words. Subsequently, a three-level model—incorporating sentence-level, news-level, and topic-level data—is employed to extract events and calculate their probabilities. Experimental results across four event types demonstrate the effectiveness of this multi-layered extraction strategy.
Introduction
The digital age has led to an explosion of unstructured news text, making programmatic analysis difficult. To facilitate data mining and retrieval, this text must be converted into structured "event triples" (Subject-Predicate-Object). However, extraction is challenging due to context dependency; a single sentence often lacks the necessary information to disambiguate meaning. The authors aim to bridge this gap by moving beyond simple keyword matching to a multi-layered probabilistic approach.
The Three-Level Extraction Framework
The authors propose a bottom-up extraction method based on three hierarchical levels:
- Sentence Level: The system identifies "trigger words" (usually verbs) to find candidate sentences. It calculates the probability of an event based on the co-occurrence of parts of speech (like names, locations, and times) within that specific sentence.
- News Level: This layer looks at the entire news report. It assumes that the more frequently event-related words appear throughout the whole text, the higher the probability that the extracted event is the "main" event of the article.
- Topic Level: Using Latent Dirichlet Allocation (LDA), the model identifies the general theme of the news (e.g., sports vs. politics). Since certain events are more likely to occur within specific topics, this provides a thematic "sanity check" for the extraction.
The final probability of an event is the product of these three levels. To simplify the process, the authors focused on extracting events contained within a single sentence rather than trying to stitch together details spread across multiple paragraphs.
Datasets and Lexicons
The study utilized a substantial corpus of Chinese media, including approximately 380,000 reports from 30 mainstream Chinese daily newspapers. A key innovation was using 20,000 news titles as training data; since titles are essentially pre-written event summaries, they provide high-quality training data without manual labeling. To expand the system's vocabulary, the authors integrated the HIT-CIR Tongyici Cilin (a Chinese synonym forest) to refine trigger word recognition, improving the system's ability to recognize different ways of describing the same action.
Key Results
The model was evaluated across four categories based on "Right rate" (Precision) and "Recall rate":
| News Type | Right Rate (Precision) | Recall Rate |
|---|---|---|
| Political News | 83.3% | 70.2% |
| Economic News | 81.5% | 71.2% |
| Sports News | 72.1% | 60.1% |
| Entertainment News | 70.2% | 65.3% |
The results show that the three-level model is highly effective for formal news categories. The authors noted that political and economic writing is often more formulaic and structured, whereas sports and entertainment writing uses more creative metaphors that are harder for a statistical model to pin down.
Limitations and Future Work
The authors identify three main areas for improvement. First, the current model lacks syntax analysis, relying on a "word-bag" approach that may struggle with complex sentences. Second, the number of topics for the LDA model was selected manually; future iterations aim to automate this selection. Finally, the authors plan to redesign the "News Level" model to move beyond simple word frequency for better accuracy.
Summary updated