Abstract
Automatic topic segmentation is the process of dividing a continuous stream of speech into topically coherent units. Traditional methods rely heavily on lexical cues—the specific words used—which are often compromised by Automatic Speech Recognition (ASR) errors. This paper presents a probabilistic framework that integrates lexical information with prosodic cues, such as pitch and pause duration. By combining a Hidden Markov Model (HMM) for lexical analysis with a Classification and Regression Tree (CART) for prosodic features, we demonstrate a significant reduction in segmentation error. Our results on the Broadcast News corpus show that prosodic features are not only competitive on their own but also provide essential robustness when ASR accuracy is low.
Introduction
Processing raw audio from broadcast news presents a significant challenge for natural language processing systems. Unlike written text, raw ASR transcripts lack punctuation, paragraph breaks, and capitalization. To make this data useful for information retrieval or summarization, the system must first perform topic segmentation: identifying the boundaries where one story ends and another begins.
While lexical models—which look for shifts in vocabulary—are standard, they face two primary hurdles. First, they are sensitive to ASR word error rates. Second, they often miss the structural cues inherent in human speech. Humans naturally use prosody—the rhythm, stress, and intonation of speech—to signal transitions. This research explores how these non-lexical signals can be modeled and fused with traditional word-based methods to create a more resilient segmenter.
Methodology
Pre-segmentation: The "Chopping" Phase
Before classification, the continuous audio stream must be divided into manageable units, or "pseudo-sentences." The researchers evaluated several criteria for these initial cuts, including fixed word counts and speaker turns. They found that the most effective method was based on silence: using pauses longer than 0.575 seconds as potential boundary candidates. This approach ensures that the system evaluates boundaries at points where a transition is naturally likely to occur.
Modeling Knowledge Sources
The system utilizes two distinct models to evaluate each potential boundary:
- Lexical Model: An HMM-based approach that treats topics as hidden states. It uses unigram language models to detect shifts in the distribution of words. When the vocabulary changes significantly, the model assigns a higher probability to a topic boundary.
- Prosodic Model: A CART-style decision tree that analyzes the acoustic properties of the speech. Key features include pause duration, F0 (fundamental frequency) resets—where a speaker starts a new topic at a higher pitch—speaker gender, and the duration of the preceding turn.
Model Integration
The core innovation of this work lies in the fusion of these two streams. Two methods were tested:
- Feature-level fusion: The probability output from the lexical HMM is fed into the prosodic decision tree as an additional feature.
- Model-level fusion: The decision tree provides posterior probabilities for boundary states, which are then integrated into the HMM's transition probabilities. This allows for global optimization across the entire sequence using the Viterbi algorithm.
Experimental Setup
The models were trained and tested using the 1997 LDC Broadcast News (BN) corpus, comprising approximately 70 hours of training data and 3 hours of test data. Performance was measured using the DARPA-TDT (Topic Detection and Tracking) metrics, which calculate a weighted cost based on "misses" (failing to identify a boundary) and "false alarms" (identifying a boundary where none exists).
Results and Discussion
The Power of Prosody
One of the most striking findings was that the prosodic model alone performed competitively with the word-based HMM. While lexical models are often superior in clean text environments, prosody proved to be a powerful signal in spoken language. Pause duration was identified as the most critical prosodic feature, though pitch resets were essential for distinguishing between simple sentence breaks and true topic shifts.
Synergy through Fusion
The combined model outperformed either individual source. By integrating prosody, the system achieved a 24% to 27% relative improvement in segmentation cost over the lexical-only baseline. The lexical model effectively acted as a filter for the prosodic model; while prosody is excellent at finding potential breaks, it often produces false alarms at sentence boundaries. The lexical model helps confirm if those breaks actually correspond to a change in subject matter.
Robustness to ASR Noise
To test real-world utility, the researchers compared performance on "true" transcripts versus ASR transcripts with a 30.5% word error rate. While the lexical-only model's performance degraded as ASR quality dropped, the combined model remained remarkably stable. Because prosodic features are extracted directly from the waveform, they remain accurate even when the speech recognizer misidentifies the words being spoken.
Conclusion
This research demonstrates that "how things are said" is just as important as "what is said" for automatic topic segmentation. By fusing prosodic and lexical cues within a probabilistic framework, we can create segmentation systems that are more accurate and significantly more robust to the errors inherent in speech recognition. Future work will focus on making these models more generalizable across different broadcast styles and exploring additional acoustic features like speaking rate and energy patterns.
Summary updated