Lexical Story Co-Segmentation of Chinese Broadcast News

Wei Feng, Xuecheng Nie, Liang Wan, Lei Xie, Jianmin JiangInterspeech 2012 · OpenReview

Abstract

Identifying shared topics across unsegmented media streams is a critical task in Topic Detection and Tracking (TDT). This paper presents an unsupervised approach for the co-segmentation of Chinese broadcast news transcripts. By formulating the problem within a Markov Random Field (MRF) framework, the proposed system iteratively refines foreground and background models to extract common stories across document pairs. Evaluation on a real-world CCTV corpus with varying levels of speech recognition noise demonstrates significant performance improvements over traditional story-matching baselines, particularly when utilizing subword lexical representations.

Introduction

In the domain of Topic Detection and Tracking (TDT), supervised systems often struggle with the dynamic nature of broadcast news, where new and unknown stories emerge continuously. A significant challenge arises when processing unsegmented transcripts: how can we automatically identify and extract segments from two different sources that discuss the same news event? This task is complicated by the inherent noise in Automatic Speech Recognition (ASR) and the lack of explicit story boundaries. This paper addresses these issues by treating the task as a co-segmentation problem—a concept adapted from computer vision—where the goal is to simultaneously segment and match related content across multiple text sources.

Methodology: Markov Random Field Co-Segmentation

The core of the proposed approach is a four-step iterative algorithm built upon a Markov Random Field (MRF) framework. This framework allows for the integration of local sentence-level coherence and global document-level similarity.

Overview of the four-step iterative co-segmentation algorithm.

The process begins with Graph Formulation & Initialization, where transcripts are represented as graphs with sentences as nodes. Edges are established both within a document (intra-doc) to maintain narrative flow and across documents (inter-doc) to link similar content. Initial "foreground" labels are assigned via lexical clustering. In the Model Refinement phase, statistical distributions for foreground and background content are updated based on current labels.

An Energy Function is then constructed to balance three critical objectives: the likelihood of a sentence belonging to its assigned model, intra-document coherence, and inter-document similarity. Finally, Hybrid Optimization is employed to minimize this energy function, utilizing Quadratic Pseudo-Boolean Optimization (QPBO) and Belief Propagation (BP) to handle the complex labeling task.

Experimental Evaluation

The system was evaluated using a CCTV corpus comprising 71 news episodes, totaling approximately 27 hours of Mandarin broadcast news. To assess robustness against real-world conditions, transcripts were generated with three different ASR accuracy rates: 59%, 66%, and 75%. The researchers tested various lexical units, including full words and subword n-grams (unigrams through quadgrams) at both the character and syllable levels.

Results and Discussion

The MRF-based co-segmentation method consistently outperformed the "story-matching" baseline across all ASR quality levels.

Comparison of F1-measure performance between the proposed co-segmentation method and the story-matching baseline across different lexical units.

Key findings from the experimental results include:

  • Subword Superiority: In the context of noisy Chinese ASR transcripts, subword units like character bigrams and quadgrams proved more effective than full words. These units are more resilient to Out-of-Vocabulary (OOV) errors common in speech recognition.
  • Performance Gains: The method achieved significant improvements, including a reported 37.5% relative gain in F1-measure using character unigrams under specific test conditions.
  • Global-Local Balance: The MRF approach successfully combines local transitions with global similarity, resulting in cleaner and more accurate story boundaries compared to local-only methods.

Conclusion

This work demonstrates the efficacy of unsupervised co-segmentation for processing unsegmented broadcast news. While the current pairwise model shows strong results, future work will focus on scaling the framework to handle larger document sets and incorporating semantic embeddings to move beyond purely lexical dependencies. Additionally, exploring dynamic sentence boundaries could further improve the precision of the extracted segments.

Summary updated

BibTeX
@inproceedings{feng2012lexical,
  author = {Wei Feng and Xuecheng Nie and Liang Wan and Lei Xie and Jianmin Jiang},
  title = {{Lexical Story Co-Segmentation of Chinese Broadcast News}},
  booktitle = {Interspeech},
  year = {2012},
  url = {https://openreview.net/forum?id=Dgh-dXdokFN},
}
Figure