Sibilant speech detection in noise

Sira Gonzalez, Mike BrookesInterspeech 2012

TL;DR

This paper presents a novel algorithm for detecting sibilant speech sounds (like "s" and "sh") in noisy audio. It achieves over 80% accuracy even at 0 dB signal-to-noise ratio by focusing on sustained increases in spectral energy rather than precise onset/offset detection, making it robust for applications like speech enhancement.

What Problem Does This Paper Address?

Identifying specific speech sounds, particularly unvoiced ones like sibilants, in the presence of background noise is a significant challenge. Sibilants carry a lot of high-frequency energy, and their accurate detection is crucial for many speech processing tasks, including speech recognition and enhancement. Existing methods often rely on detecting the precise start and end of these sounds or on identifying the fundamental frequency of speech, which can be unreliable in noisy conditions. This paper aims to develop a more robust method for sibilant detection that doesn't depend on precise timing or fundamental frequency estimation, making it more effective in noisy environments.

Methods

The proposed algorithm operates in the frequency domain and uses a two-stage approach:

  1. Sibilant Speech Energy Estimation: Instead of pinpointing sibilant onsets and offsets, the algorithm looks for a sustained increase in power within specific frequency bands over a time frame. It models the noisy speech signal using complex Gaussian random variables for both speech and noise. By maximizing the log-likelihood of observing the signal under the assumption of a sibilant's presence, it estimates the mean power of the noise ($a_k$) and the sibilant ($b_k$) in each frequency band. To account for varying sibilant durations, a weighting function is applied to the log-likelihood calculation, giving less importance to frames at the edges of potential sibilant segments. A maximum filter is then applied to the estimated sibilant power to ensure a high-level estimate across the sibilant's duration. Finally, this estimate is normalized to be independent of the overall speech level.
  2. Classification using Gaussian Mixture Models (GMMs): The normalized sibilant power spectrum for each frame serves as input to two GMMs. One GMM is trained on non-sibilant speech frames, and the other is trained on sibilant speech frames. The likelihood ratio between these two models is then used to classify each frame as either sibilant or non-sibilant.

Datasets

  • TIMIT database: Used for both training and evaluation. The paper notes that phonetic transcriptions within TIMIT were used for training the GMMs.

Key Insights and Ideas

  • Sustained Energy Increase as a Sibilant Cue: The core idea that a sustained increase in spectral energy, rather than precise temporal boundaries, is a robust indicator of sibilant presence is a key insight. This decouples the detection from the difficult task of exact onset/offset localization.
  • Robustness to Noise: By designing the estimation and classification stages to be less sensitive to precise timing and by training GMMs at a 0 dB SNR, the algorithm demonstrates significant robustness to high levels of noise.
  • Frequency Band Analysis: Utilizing 14 frequency bands from 1.5 kHz to 8 kHz allows for a detailed spectral characterization of sibilants, which are rich in high-frequency energy.
  • Weighting for Duration Variability: The use of a weighting function to accommodate the natural variation in sibilant durations is a practical approach to improve estimation accuracy.

Key Results

  • The algorithm consistently achieved over 80% classification accuracy for positive SNRs on the TIMIT test set.
  • At a 0 dB signal-to-noise ratio (SNR), the equal error rate (where miss probability equals false alarm probability) was 16.5%, indicating an 83.5% classification accuracy.
  • Even at -5 dB SNR, the algorithm maintained a reasonable performance, with a miss probability of 33.7% and a false alarm probability of 10.9% at a unity likelihood ratio threshold.
  • The paper highlights that miss rates increase with decreasing SNR as sibilants become more masked by noise, while false alarm rates remain relatively low.

Limitations and Future Work

  • Frame-by-Frame Classification: The paper acknowledges that classifying each frame individually might limit performance. Applying temporal constraints to the classification decisions could potentially improve accuracy.
  • Manual Parameter Tuning: While parameters were determined using the TIMIT training set, the paper doesn't explicitly detail the process of finding optimal values for all parameters (e.g., sibilant duration S, weighting function parameters, maximum filter width W).
  • Future Work: The authors suggest that incorporating temporal constraints is a promising direction for future improvement.

Summary updated

BibTeX
@inproceedings{gonzalez2012sibilant,
  author = {Sira Gonzalez and Mike Brookes},
  title = {{Sibilant speech detection in noise}},
  booktitle = {Interspeech},
  year = {2012},
}
Figure