TL;DR
This paper introduces a novel approach to automatically detect and track news events from a continuous stream of multilingual news articles. By combining concept terms, named entities, and traditional story keywords, their system achieves improved event detection performance compared to keyword-only methods, particularly in handling the complexities of multilingual news.
What Problem Does This Paper Address?
The ever-increasing volume of news content stored digitally presents a significant challenge for information retrieval. Traditional query-driven systems are effective when users know exactly what they're looking for, but they falter when dealing with broad queries like "What happened?" or when users need to catch up on events after a long absence. Manually sifting through vast archives is impractical. This paper tackles the problem of automatic event detection, aiming to build an intelligent system that can identify and group topically related news stories, even across different languages, without predefined event categories. This is crucial for applications like temporal event evolution tracking and content shift analysis.
Methods
The core of the proposed method lies in a sophisticated story representation that goes beyond simple keyword matching. Each story is represented by three types of vectors:
- Concept Feature Vector: This vector captures the underlying concepts of a story. Concept terms are derived by analyzing the statistical context between sentences in the incoming story and a separate "concept database" built from a large corpus of news stories. This helps overcome vocabulary switching issues.
- Named Entity Feature Vector: This vector represents named entities (persons, locations, organizations) extracted from the story using a transformation-based error-driven part-of-speech tagger.
- Story Feature Vector: This vector represents the traditional keywords of the story, with weights adjusted to prioritize terms appearing in earlier, more summary-like sentences.
For Chinese stories, a gross translation method is employed to convert Chinese terms into English equivalents, enabling a unified representation for multilingual analysis.
The system then uses a modified agglomerative clustering algorithm to group similar stories. This algorithm incorporates "temporary clusters" to handle the incremental nature of news streams and a time-adjustment scheme to account for the temporal proximity of stories when calculating similarity. The final similarity score between stories (or stories and events) is a weighted combination of concept relevance, named entity relevance, and story feature relevance.
Datasets
The experiments were conducted using the official development corpus of the Topic Detection and Tracking (TDT) project from the Linguistic Data Consortium (LDC). This corpus was annotated for 100 topics and contained news from various English and Chinese sources, including:
- English sources: New York Times Newswire Service, Associated Press Worldstream Service, Cable News Network, American Broadcasting Company, Public Radio International, Voice of America.
- Chinese sources: Xinhua News Agency, Zaobao News Agency, Voice of America.
The first four months of 1998, comprising 43,666 stories, were used for the experiments.
Key Insights and Ideas
- Beyond Keywords: The paper strongly advocates for moving beyond simple keyword matching in event detection. Incorporating concept terms and named entities provides a richer, more robust representation of story content.
- Concept Terms for Vocabulary Switching: The use of concept terms, derived from a separate concept database, is a key innovation for handling vocabulary switching – where different words are used to describe the same event. This allows the system to connect semantically related stories even if they use disparate terminology.
- Multilingual Handling: The gross translation approach for Chinese stories is a practical solution for integrating multilingual news into a unified detection framework, enabling cross-lingual event detection.
- Time-Aware Similarity: The introduction of a time-adjustment scheme in similarity calculation is crucial for event detection, as stories that occurred far apart in time are less likely to belong to the same event.
- Combined Feature Strength: The experimental results highlight that combining multiple feature types (concept terms, named entities, story terms) leads to significantly better detection performance than relying on any single feature type alone.
Key Results
The paper reports that their proposed method, using a combination of concept terms, named entities, and story terms, outperforms traditional keyword-only methods. Specifically:
- The best detection performance, measured by a normalized detection cost (Cdet)norm, was achieved with a threshold (h) of 0.14, resulting in a Cdet)norm of 0.3248.
- Experiments showed that using concept terms significantly improves detection performance, leading to a better characterization of stories and a more accurate event detection. Figure 7 illustrates this improvement, showing a lower Cdet)norm when concept terms are included.
- The system generally achieves a relatively small false alarm probability, though this can sometimes come at the cost of higher miss probabilities.
Limitations and Future Work
The authors acknowledge several limitations and propose future research directions:
- Translation Module Improvement: Developing a more powerful translation module that can translate Chinese words into more meaningful English terms is a key area for future work.
- Incremental Inverse Document Frequency: Investigating the use of incremental inverse document frequency (IDF) could further enhance retrieval performance.
- Sophisticated Clustering: Developing a more sophisticated clustering algorithm is suggested as a way to reduce miss probabilities.
- Parameter Tuning: While experiments were conducted on tuning parameters like the threshold
hand weightsw_s,w_c,w_n, further exploration of parameter optimization could yield even better results.
Summary updated