Abstract
Identifying shared topics across unsegmented media streams is a critical task in Topic Detection and Tracking (TDT). This paper presents an unsupervised approach for the co-segmentation of Chinese broadcast news transcripts. By formulating the problem within a Markov Random Field (MRF) framework, the proposed system iteratively refines foreground and background models to extract common stories across document pairs. Evaluation on a real-world CCTV corpus with varying levels of speech recognition noise demonstrates significant performance improvements over traditional story-matching baselines, particularly when utilizing subword lexical representations.
Introduction
In the domain of Topic Detection and Tracking (TDT), supervised systems often struggle with the dynamic nature of broadcast news, where new and unknown stories emerge continuously. A significant challenge arises when processing unsegmented transcripts: how can we automatically identify and extract segments from two different sources that discuss the same news event? This task is complicated by the inherent noise in Automatic Speech Recognition (ASR) and the lack of explicit story boundaries. This paper addresses these issues by treating the task as a co-segmentation problem—a concept adapted from computer vision—where the goal is to simultaneously segment and match related content across multiple text sources.
Methodology: Markov Random Field Co-Segmentation
The core of the proposed approach is a four-step iterative algorithm built upon a Markov Random Field (MRF) framework. This framework allows for the integration of local sentence-level coherence and global document-level similarity.
The process begins with Graph Formulation & Initialization, where transcripts are represented as graphs with sentences as nodes. Edges are established both within a document (intra-doc) to maintain narrative flow and across documents (inter-doc) to link similar content. Initial "foreground" labels are assigned via lexical clustering. In the Model Refinement phase, statistical distributions for foreground and background content are updated based on current labels.
An Energy Function is then constructed to balance three critical objectives: the likelihood of a sentence belonging to its assigned model, intra-document coherence, and inter-document similarity. Finally, Hybrid Optimization is employed to minimize this energy function, utilizing Quadratic Pseudo-Boolean Optimization (QPBO) and Belief Propagation (BP) to handle the complex labeling task.
Experimental Evaluation
The system was evaluated using a CCTV corpus comprising 71 news episodes, totaling approximately 27 hours of Mandarin broadcast news. To assess robustness against real-world conditions, transcripts were generated with three different ASR accuracy rates: 59%, 66%, and 75%. The researchers tested various lexical units, including full words and subword n-grams (unigrams through quadgrams) at both the character and syllable levels.
Results and Discussion
The MRF-based co-segmentation method consistently outperformed the "story-matching" baseline across all ASR quality levels.
Key findings from the experimental results include:
- Subword Superiority: In the context of noisy Chinese ASR transcripts, subword units like character bigrams and quadgrams proved more effective than full words. These units are more resilient to Out-of-Vocabulary (OOV) errors common in speech recognition.
- Performance Gains: The method achieved significant improvements, including a reported 37.5% relative gain in F1-measure using character unigrams under specific test conditions.
- Global-Local Balance: The MRF approach successfully combines local transitions with global similarity, resulting in cleaner and more accurate story boundaries compared to local-only methods.
Conclusion
This work demonstrates the efficacy of unsupervised co-segmentation for processing unsegmented broadcast news. While the current pairwise model shows strong results, future work will focus on scaling the framework to handle larger document sets and incorporating semantic embeddings to move beyond purely lexical dependencies. Additionally, exploring dynamic sentence boundaries could further improve the precision of the extracted segments.
Summary updated