Semi-supervised extractive speech summarization via co-training algorithm

Shasha Xie, Hui Lin, Yang Liu

Abstract

Extractive summarization of spontaneous speech, such as meeting recordings, is hindered by the high cost of manual labeling. This paper investigates a semi-supervised co-training approach that leverages the natural duality of speech data—textual and acoustic features—to improve summarization performance with limited labeled data. Our results on the ICSI Meeting Corpus demonstrate that sentence-based co-training significantly outperforms supervised baselines, particularly in low-resource scenarios, and helps bridge the performance gap between textual and acoustic models.

Introduction

The proliferation of digital meeting recordings has created a pressing need for automated summarization tools. Extractive summarization, which identifies and concatenates salient sentences from a transcript, is a common approach. However, applying this to speech is challenging due to the disfluent nature of spontaneous dialogue and the "data bottleneck": creating gold-standard summaries for training requires intensive human effort.

To address this, we explore semi-supervised learning, specifically the co-training algorithm. By utilizing the vast amounts of available unlabeled meeting data, we aim to enhance model robustness without the need for exhaustive manual annotation.

Methodology: The Co-Training Framework

Co-training relies on the existence of two independent and sufficient "views" of the data. Speech summarization is uniquely suited for this because each utterance possesses both textual (lexical content, syntax) and acoustic (prosodic features like pitch and energy) characteristics.

The Co-Training Loop

The process begins with a small set of labeled meetings. Two separate classifiers are trained: one on textual features and another on acoustic features. These models then iterate through a large pool of unlabeled data:

  1. Each classifier predicts the importance of sentences in the unlabeled set.
  2. The most confident predictions are selected and added to the training set of the other classifier.
  3. The models are retrained, and the cycle repeats.

We compared two selection strategies: Sentence-based selection, where individual high-confidence sentences are added, and Document-based selection, where entire meetings with high average confidence are added.

Experimental Setup

We evaluated our approach using the ICSI Meeting Corpus, consisting of 75 natural meetings. Performance was measured using the ROUGE-1 F-measure, comparing system-generated extracts against human-authored summaries. We tested the models across varying amounts of initial labeled data, from 10% to 90% of the training set.

Results and Analysis

Performance Gains in Low-Resource Regimes

The primary advantage of co-training is observed when labeled data is most scarce. As shown in Figure 2, the sentence-based co-training approach provides a substantial boost over the supervised baseline when only 10% to 30% of the data is labeled.

Figure 2: ROUGE-1 performance of the textual-feature model. The sentence-based co-training (red) significantly outperforms the baseline (black) in low-data scenarios.

Notably, the co-training process allowed the initially "weaker" textual model to learn from the more confident acoustic model, effectively closing the performance gap between the two views.

Sentence vs. Document Selection

Our experiments revealed a clear preference for sentence-level granularity in the selection process. While document-based selection was hypothesized to provide more context, it often introduced excessive noise.

Figure 4: Comparison of selection strategies. Sentence-based co-training consistently maintains a lead over document-based selection and the baseline.

As illustrated in Figure 4, document-based selection (blue line) frequently underperformed compared to sentence-based selection (red line) and occasionally fell below the supervised baseline. This suggests that high confidence in a document's overall summary does not guarantee the accuracy of every individual sentence label within it.

Conclusion

This study demonstrates that co-training is a highly effective semi-supervised strategy for speech summarization, exploiting the natural split between text and audio. Sentence-based selection is the superior strategy for expanding the training set. Future work will investigate more nuanced selection criteria and the application of other semi-supervised techniques like graph-based learning to further mitigate the data bottleneck in speech processing tasks.

Summary updated

BibTeX
@inproceedings{xie2010semi,
  author = {Shasha Xie and Hui Lin and Yang Liu},
  title = {{Semi-supervised extractive speech summarization via co-training algorithm}},
  booktitle = {Interspeech},
  year = {2010},
  doi = {10.21437/interspeech.2010-677},
}
Figure