Sparse component analysis for speech recognition in multi-speaker environment

Afsaneh Asaei, Hervé Bourlard, Philip N. GarnerInterspeech 2010 · OpenReview

Abstract

This paper explores the application of Sparse Component Analysis (SCA), specifically the Degenerate Unmixing Estimation Technique (DUET), to the problem of automatic speech recognition (ASR) in overlapping speech scenarios. By leveraging the sparsity of speech in the time-frequency domain, the authors demonstrate significant word accuracy improvements in under-determined environments where speakers outnumber microphones.

Introduction

Automatic Speech Recognition (ASR) systems frequently fail in "cocktail party" environments. While humans can easily focus on a single voice amidst background noise, machines struggle when speech segments overlap. In typical meetings or phone calls, approximately 10–15% of words overlap, which can lead to a 15–30% absolute increase in Word Error Rate (WER).

Traditional solutions like beamforming or Independent Component Analysis (ICA) typically require at least as many microphones as there are speakers. This research addresses the under-determined problem—where speakers outnumber microphones—by exploiting the mathematical property of sparsity through Sparse Component Analysis (SCA).

Methodology

The authors employ the Degenerate Unmixing Estimation Technique (DUET), which relies on the "W-disjoint orthogonality" assumption. This principle suggests that in the time-frequency (spectrogram) domain, it is highly improbable for two speakers to have significant energy at the exact same time and frequency coordinates.

Key technical contributions include:

  • Subband Weighted Histograms: To avoid "phase wrapping" errors during speaker localization, the researchers focused on a specific subband (100–570 Hz) that captures fundamental pitch and first formants.
  • Hyper-parameter Optimization: A grid search was conducted to determine optimal weighting parameters ($p=0.5, q=1$) for the mixing model.
  • SNR-based Spectral Smoothing: Masking interfering speech often leaves "holes" in the resulting spectra, which confuses standard ASR features like MFCCs. To mitigate this, the authors used an Overlap-Add (OLA) reconstruction method and introduced a small amount of white Gaussian noise (37 dB SNR) to smooth the spectral shape for the recognizer.

Experimental Setup

The researchers created a multi-speaker environment by mixing clean utterances from the Aurora 2 benchmark with interfering sentences from the HTIMIT corpus. This simulated a physical room setup using only two microphones to capture multiple sources.

Experimental setup for multi-speaker speech separation using a two-microphone array.

Key Insights

  • Sparsity as a Foundation: The authors verified that speech fits a power-law decay model for sparsity. This confirms that the salient information required for recognition is preserved in a small fraction of the total time-frequency components.
Verification of speech sparsity showing power-law decay of coefficients.
  • Noise as a Buffer: Counter-intuitively, adding a small amount of white noise to the separated signal improved recognition. The noise acts as a buffer that smooths the sharp edges in the spectrogram created by the separation process.
  • Spatial Resolution: The system performed better with "far-microphones" (30cm apart) than "close-microphones" (5cm apart). The increased delay between microphones made it easier to distinguish different spatial signatures in the histogram.

Results

The performance gains were dramatic, particularly when using Multi-Condition training:

  • Baseline Accuracy: In a 30cm microphone setup with overlapping speech, the standard Aurora 2 baseline achieved only 44% accuracy.
  • DUET + OLA: Separation alone increased accuracy to 79.95%.
  • DUET + OLA + Noise: Adding spectral smoothing pushed accuracy to 93.35%.
  • Scalability: The system maintained a recognition rate above 70% even when the number of simultaneous speakers was increased to 10.

Conclusion

While the current model assumes an anechoic (echo-free) environment, the results prove that SCA can effectively handle under-determined speech separation for ASR. Future work will focus on identifying better "dictionaries" (such as Gabor atoms) to improve performance in highly reverberant, real-world environments.

Summary updated

BibTeX
@inproceedings{asaei2010sparse,
  author = {Afsaneh Asaei and Hervé Bourlard and Philip N. Garner},
  title = {{Sparse component analysis for speech recognition in multi-speaker environment}},
  booktitle = {Interspeech},
  year = {2010},
  url = {https://openreview.net/forum?id=RS3bUNHg22M},
}
Figure