Overlapped speech detection in meeting using cross-channel spectral subtraction and spectrum similarity

Ryo Yokoyama, Yu Nasu, Koichi Shinoda, Koji IwanoInterspeech 2012

TL;DR

This paper tackles the challenge of accurately detecting when multiple people are speaking simultaneously in meeting recordings. They introduce two novel features, "CCSS power" and "amplitude spectral cosine correlation coefficient (ACC)," which significantly improve detection accuracy compared to previous methods, paving the way for better meeting transcription and analysis.

What Problem Does This Paper Address?

Accurately transcribing and analyzing meeting recordings is crucial for many applications, but a major hurdle is dealing with "overlapped speech" – moments when two or more people are talking at the same time. This overlap makes it difficult for speech recognition systems to isolate individual speakers and can lead to garbled or missed information. While previous methods have tried to address this using features like raw speech power and spectral similarity, they often struggle with noise and normalization issues that can reduce their effectiveness, especially in relatively quiet meeting environments. This paper aims to improve overlapped speech detection (OSD) by developing more robust and effective features.

Methods

The authors propose a GMM-based classifier for OSD, but their main contribution lies in the novel features they use as input:

  • CCSS Power (Cross-Channel Spectral Subtraction Power): This feature is derived from a source separation technique called CCSS. It aims to reduce the contribution of other speakers' voices from a given microphone's signal. By estimating and subtracting the power of other speakers, the CCSS power focuses on the target speaker's contribution. The implementation simplifies this by assuming the target speaker's voice is dominant and using a simplified subtraction.
  • Amplitude Spectral Cosine Correlation Coefficient (ACC): This feature measures the similarity between the amplitude spectra of different microphones. Unlike previous methods that used power spectra and normalization (which can sometimes suppress actual speech signals), ACC uses amplitude spectra and avoids normalization. The authors hypothesize this will be more effective in the relatively quiet conditions of meeting speech.

These two features are then fed into a Gaussian Mixture Model (GMM) classifier, which is trained to distinguish between overlapped and non-overlapped speech segments.

Datasets

The experiments were conducted using a newly recorded sit-down meeting dataset of 19 minutes, featuring four speakers (one female, three male) speaking Japanese. The participants wore lapel microphones. The dataset was hand-labeled for overlapped speech (W) and non-overlapped speech (Wn) at a 10 ms frame rate. The dataset was split into training (70%) and testing (30%) sets.

Key Insights and Ideas

  • The limitations of traditional features: Raw speech power can be easily contaminated by other speakers, and normalized spectral similarity (like PPC) can inadvertently suppress signals from a single speaker in quiet environments.
  • Complementary nature of new features: CCSS power is good at isolating the target speaker's signal by reducing others, while ACC is good at capturing spectral similarity without problematic normalization. Together, they offer a more robust detection.
  • Amplitude vs. Power for Similarity: Using amplitude spectra for similarity (ACC) instead of power spectra (PPC) is beneficial because it maintains a smaller dynamic range, potentially leading to better performance in meeting scenarios.

Key Results

The proposed method, using CCSS power and ACC (CCSS_P+ACC), achieved a significant improvement in detection performance.

  • Overall Accuracy: The proposed method achieved an Average Precision (AP) of 74.1%, which is a substantial improvement over the previous method (P+PPC) that achieved 67.0% AP. This represents a 10.6% relative improvement.
  • Feature Effectiveness: Visualizations (scatter plots and histograms) showed that CCSS_P and ACC individually and combined provide better separation between overlapped and non-overlapped speech segments compared to previous features. Specifically, CCSS_P helps separate regions with higher power, while ACC helps with regions of lower power, making them complementary.

Limitations and Future Work

  • Misdetected Frames: Despite improvements, misdetected frames still exist. The authors suggest that incorporating additional features, such as entropy, could further enhance performance.
  • Hand-Labeled Data: The current study relies on hand-labeled speech segments. For practical applications, developing an overlapped detection method that uses unsupervised learning would reduce the significant annotation costs.

Summary updated

BibTeX
@inproceedings{yokoyama2012overlapped,
  author = {Ryo Yokoyama and Yu Nasu and Koichi Shinoda and Koji Iwano},
  title = {{Overlapped speech detection in meeting using cross-channel spectral subtraction and spectrum similarity}},
  booktitle = {Interspeech},
  year = {2012},
}
Figure