Automatic metric-based speech segmentation for broadcast news via principal component analysis

Jeih-Weih Hung, Hsin-Min Wang, Lin-Shan LeeInterspeech 2000 · DOI

TL;DR

This paper introduces a novel approach to automatically segmenting broadcast news audio by first applying Principal Component Analysis (PCA) to speech features. This pre-processing step makes the features more robust, leading to a significant improvement in the accuracy of metric-based segmentation techniques, particularly in noisy or distorted audio.

What Problem Does This Paper Address?

Accurately segmenting audio signals, especially broadcast news, is crucial for downstream tasks like speech recognition, transcription, and retrieval. Existing metric-based segmentation methods rely on measuring the acoustic distance between consecutive audio windows. However, their performance is heavily influenced by the quality of the speech features used. Standard features like MFCCs can be easily corrupted by noise or channel distortion, hindering the effectiveness of these segmentation techniques. This paper aims to improve the robustness and accuracy of metric-based segmentation by enhancing the speech features themselves.

Methods

The core of this paper's contribution lies in its application of Principal Component Analysis (PCA) to speech features before applying metric-based segmentation. PCA is a dimensionality reduction technique that identifies the principal components (directions of maximum variance) in a dataset. By projecting the original speech features onto a subspace defined by the top k eigenvectors of the covariance matrix, the authors create more discriminative and robust "PCA-derived features."

They propose two specific approaches for generating these PCA-derived features:

  1. Two Eigenspaces Approach: For each pair of contiguous windows, separate PCA is performed. Each window's features are then projected onto its own respective PCA subspace.
  2. One Eigenspace Approach: A single common eigenspace is derived from the combined features of both contiguous windows. Both windows' features are then projected onto this shared subspace.

After obtaining these PCA-derived features, the standard metric-based segmentation process is applied. This involves modeling the features within each window (e.g., using Gaussian distributions) and calculating a distance between these models. Peaks in the resulting distance curve are identified as candidate segmentation points.

Datasets

The experimental data consisted of 29 sections of broadcast news reports from CTS (Chinese Television System in Taiwan), totaling approximately 46 minutes of audio. The audio was recorded at an 8K Hz sampling rate. The ground truth segmentation was based on speaker changes, environmental changes, and silence periods, resulting in 1023 segmentation points.

Key Insights and Ideas

  • PCA as a Feature Enhancer for Segmentation: The most significant insight is that PCA can effectively "clean up" and make speech features more discriminative for segmentation, especially in challenging acoustic conditions.
  • Robustness to Noise and Distortion: By capturing the most significant variance in the data, PCA-derived features are inherently less sensitive to noise and channel distortions that plague raw MFCCs.
  • Trade-off in Eigenspace Strategy: The "Two Eigenspaces" approach generally outperformed the "One Eigenspace" approach. This is attributed to the fact that using separate eigenspaces for each window better preserves the acoustic differences between them, especially when they represent distinct environments. A single common eigenspace can sometimes "dilute" these differences.
  • Fewer Eigenvectors Often Better: In most cases, using a smaller number of eigenvectors (i.e., a lower-dimensional PCA subspace) led to better detection rates. This suggests that the most expressive components of the speech signal for segmentation are captured by the top few principal components, and higher-order components might introduce noise or irrelevant variance. This also offers computational benefits.

Key Results

The proposed PCA-based metric-based segmentation approaches demonstrated significant improvements in detection rates compared to using raw MFCC features.

  • Detection Rate Improvement: The paper reports improvements of up to 7% in detection rates, while false alarm rates remained unchanged.
  • Distance Measure Impact: The improvements were particularly noticeable when using Mahalanobis distance or Kullback-Leibler distance measures.
  • Approach Comparison: The "Two Eigenspaces" approach generally yielded higher detection rates than the "One Eigenspace" approach across various distance measures and numbers of eigenvectors.
  • Eigenvector Count: Using fewer eigenvectors (e.g., 1 or 3) for constructing the eigenspace often resulted in better performance. For instance, with Mahalanobis distance and the "Two Eigenspaces" approach, using 1 eigenvector achieved a detection rate of 94.62%, compared to 88.86% for MFCCs.

Limitations and Future Work

  • Computational Complexity: While the paper notes that using fewer eigenvectors can reduce computation, the PCA process itself adds an initial computational overhead compared to direct MFCC usage.
  • Parameter Tuning: The optimal number of eigenvectors (k) and the specific distance measures and thresholds still require careful selection, which can be data-dependent.
  • Future Work: The authors suggest that these improvements in segmentation accuracy are expected to enhance the performance of subsequent speech signal processing and speech recognition tasks. Further exploration into different feature extraction methods and segmentation criteria could also be beneficial.

Summary updated

BibTeX
@inproceedings{hung2000automatic,
  author = {Jeih-Weih Hung and Hsin-Min Wang and Lin-Shan Lee},
  title = {{Automatic metric-based speech segmentation for broadcast news via principal component analysis}},
  booktitle = {Interspeech},
  year = {2000},
  doi = {10.21437/icslp.2000-766},
}
Figure