Introduction
In the dynamic landscape of news and social media, events rarely occur in isolation. Instead, they unfold as a progression of interconnected subevents. For instance, a natural disaster event might sequence from an initial "earthquake" to "damages" and eventually "rescue efforts." Predicting these future subevents is vital for governments, news agencies, and organizations to implement proactive measures. However, modeling this progression presents significant challenges: accurately representing subevents without manual feature engineering, capturing the nested hierarchical structure of language (words within subevents, subevents within events), and accounting for the overarching thematic context that influences event trajectories.
Methodology: The CH-LSTM Architecture
To address these challenges, we introduce the Contextual Hierarchical LSTM (CH-LSTM). This generative model moves beyond simple classification to predict the natural language content of future subevents. The architecture is built upon three primary components:
- Subevent-Level Encoder: An LSTM that processes word sequences within a single subevent (e.g., a news headline) to generate a dense vector representation.
- Event-Level Encoder: A higher-level LSTM that models the temporal sequence of subevent vectors. To incorporate global context, we utilize Latent Dirichlet Allocation (LDA) to extract topic embeddings for each subevent. These embeddings are concatenated with the subevent vectors, providing a semantic prior that helps the model distinguish between similar actions in different contexts (e.g., a "peak" in travel vs. a "peak" in energy consumption).
- Next-Subevent Decoder: A final LSTM that decodes the hidden state of the event-level encoder into a word-by-word prediction of the subsequent subevent.
By employing this hierarchical structure, the model effectively mitigates the vanishing gradient problem associated with processing long, flat sequences of text, while the contextual topic features act as semantic anchors for more accurate predictions.
Data and Experimental Setup
Due to the lack of existing benchmarks for this task, a large-scale dataset was curated from Sina News. The dataset comprises 15,254 news series, segmented into 155,358 events using a sliding window approach. The model focuses on news headlines, which serve as concise summaries of subevent content. After preprocessing, the vocabulary consisted of 4,515 unique words.
Results and Analysis
The CH-LSTM was evaluated against several baselines, including standard n-gram models, a flat LSTM, and a basic Hierarchical LSTM (HLSTM) without contextual features. Performance was measured using Perplexity and Error Rate.
As shown in the results, the transition from a flat LSTM to a hierarchical structure (HLSTM) yielded a substantial performance gain. The addition of topic-based context in the CH-LSTM further improved results, achieving a perplexity of 127.74, compared to 201.59 for the standard LSTM. Qualitatively, the model demonstrated an ability to generate semantically relevant headlines that closely matched ground truth reports in case studies involving holiday travel and system failures.
Conclusion and Future Work
The CH-LSTM demonstrates that hierarchical modeling combined with latent topic context is a powerful approach for predicting event progressions. While the model significantly reduces perplexity and improves ranking accuracy, the absolute error rate remains high, reflecting the inherent difficulty of predicting exact natural language phrasing. Future research will explore the integration of attention mechanisms to allow the model to focus on specific historical subevents and investigate how this architecture scales to full-length articles where textual noise is more prevalent.
Summary updated