Abstract
Automatic story segmentation is a fundamental task for indexing and retrieving information from continuous media streams like broadcast news. This paper introduces a hybrid architecture that combines the temporal modeling of Hidden Markov Models (HMMs) with the discriminative power of Deep Neural Networks (DNNs). By replacing generative word-count models with a DNN-based topic posterior estimator, the proposed method achieves an F1-score of 0.765 on the TDT2 corpus, representing a 20% improvement over traditional HMM baselines.
Introduction
Story segmentation involves partitioning a continuous stream of data—such as a 30-minute news broadcast—into topically coherent units. This process is essential for downstream applications like summarization and information retrieval. Historically, this task has been approached using Hidden Markov Models (HMMs) where states represent topics. However, traditional HMMs rely on generative n-gram language models that often suffer from data sparsity and fail to capture deep semantic context.
The challenge lies in creating a model that can better distinguish subtle topic transitions in noisy transcripts. This paper addresses this by moving from a generative framework to a discriminative one, leveraging DNNs to predict topic shifts more accurately.
Methodology: The Hybrid DNN-HMM Architecture
The proposed model integrates the temporal strengths of HMMs with the feature-learning capabilities of DNNs.
- HMM Framework: The HMM models the sequence of topics over time. A transition between states in the HMM indicates a story boundary.
- DNN as a Posterior Estimator: Unlike traditional generative models that calculate $P(words|topic)$, the DNN directly estimates the probability of a topic given a local window of words ($P(topic|words)$).
- Input Features: The DNN processes a Bag-of-Words (BOW) vector representing a local context window (typically 60 words) around potential boundaries.
- Unsupervised Labeling: To overcome the lack of labeled topic data, the authors employed the CLUTO clustering toolkit to group training segments into clusters. These clusters serve as "noisy" ground-truth labels for training the DNN.
- Viterbi Decoding: During inference, the Viterbi algorithm identifies the most likely sequence of topics. A story boundary is marked whenever the predicted topic changes.
Experimental Setup
The model was evaluated using the Topic Detection and Tracking (TDT2) corpus, consisting of 2,280 English broadcast news programs. The dataset was split into 1,800 programs for training, 240 for development, and 240 for testing. Text preprocessing included Porter stemming and stop-word removal, resulting in a vocabulary of 57,817 words.
Analysis and Insights
A key advantage of the DNN-HMM approach is its discriminative power. By focusing on lexical cues that differentiate topics rather than just modeling word frequencies, the DNN handles the "curse of dimensionality" more effectively than raw n-gram counts.
As shown in the similarity matrix dotplots above, the DNN-generated posteriors create significantly sharper and more distinct blocks of topical consistency compared to traditional TF-IDF or Latent Dirichlet Allocation (LDA) methods. This visual cohesion translates directly into better segmentation performance.
Results
The DNN-HMM approach significantly outperformed existing benchmarks on the TDT2 task:
- F1-Measure: The proposed model achieved an F1-score of 0.765, compared to 0.637 for traditional n-gram HMMs.
- Comparison: It also surpassed other state-of-the-art methods such as BayesSeg (0.710) and DD-CRP (0.730).
- Robustness: The performance remained stable across variations in the number of clusters (50–200) and context window sizes.
Conclusion and Future Work
While the DNN-HMM model sets a new benchmark for text-based story segmentation, several avenues for improvement remain. Currently, the model relies solely on lexical features. Future work aims to integrate multi-modal cues such as prosody, pauses, and speaker changes, which are critical indicators in broadcast media. Additionally, exploring more advanced architectures like Convolutional Neural Networks (CNNs) or Recurrent Neural Networks (RNNs) may further enhance the model's ability to capture long-term linguistic dependencies.
Summary updated