Robust phoneme recognition based on biomimetic speech contours

Michael A. Carlin, Kailash Patil, Sridhar Krishna Nemala, Mounya ElhilaliInterspeech 2012

TL;DR

This paper proposes a novel approach to robust phoneme recognition by mimicking how the human auditory system processes sound. By optimizing for "sustained neural firing," they derive a data-driven filter that effectively isolates speech modulations robust to noise and reverberation, leading to significant improvements in recognition accuracy.

What Problem Does This Paper Address?

Automatic speech recognition (ASR) systems often struggle with degraded audio, such as background noise and reverberation. While many feature extraction methods perform well in clean conditions, their performance degrades significantly in real-world scenarios. This paper addresses this challenge by drawing inspiration from the mammalian auditory system, which exhibits remarkable robustness to such degradations. Specifically, they investigate how the auditory system's neural coding strategies, particularly the concept of "sustained neural firing," can inform the design of more robust speech features.

Methods

The core of this work lies in deriving a novel 2D spectro-temporal modulation filter. This filter is designed based on the principle of optimizing "sustained firing" in a computational model of auditory neurons. Here's a breakdown:

  • Spectro-Temporal Receptive Fields (STRFs): The authors use STRFs to model the linear transformation of acoustic stimuli to neural responses.
  • Sustained Firing Criterion: They define an objective function, $J_{sus}(H)$, that quantifies how "sustained" the firing rate of an ensemble of neurons is over a given time interval. This criterion is optimized subject to constraints that ensure unit variance and mutual decorrelation of neural responses.
  • Ensemble Modulation Transfer Function (eMTF): By analyzing the modulation transfer functions of the optimized STRFs, they obtain an eMTF. This eMTF reveals the average spectro-temporal modulation sensitivity of the emergent neural ensemble.
  • Biomimetic Filter Design: The eMTF exhibits a "contouring" effect around specific modulation rates and scales that are known to be important for speech. The authors leverage these contours to design a 2D spectro-temporal modulation filter. This filter is designed to pass modulations within these identified contours while attenuating others.
  • Filtering Process: Auditory spectrograms are transformed into the modulation domain, multiplied by the designed filter, and then transformed back to the time-frequency domain.

Datasets

  • Training: Approximately three minutes of speech from the TIMIT corpus (male and female speakers) was used to learn the STRFs.
  • Testing: A separate set of 1344 utterances from the TIMIT corpus was used for testing.
  • Noise: The NOISEX92 corpus was used to generate additive noise conditions (Babble, F16, Factory1, Tank, Volvo) at various Signal-to-Noise Ratios (SNRs) from 0-20 dB.
  • Reverberation: Artificial room responses were synthesized with reverberation time constants (RT60) ranging from 100-500 ms.

Key Insights and Ideas

  • Auditory System as a Blueprint for Robustness: The paper strongly suggests that by understanding and mimicking the neural coding strategies of the auditory system, we can develop more robust speech processing techniques.
  • Sustained Firing as a Key Principle: Optimizing for sustained neural firing, a property observed in auditory neurons, leads to the emergence of receptive fields that naturally contour important speech modulations.
  • Data-Driven Filter Design: Instead of relying on hand-crafted features or predefined modulation filters, this approach derives a filter directly from the data based on neurophysiological principles.
  • Contouring Speech Modulations: The emergent STRFs, when analyzed via their eMTF, effectively "contour" the spectro-temporal modulation regions that carry significant speech information, acting as a natural boundary against noise.

Key Results

  • Significant Improvement in Phoneme Recognition: The proposed 2D filtered features consistently outperformed baseline MFCC+MVA features across all tested noise conditions and SNRs.
  • Additive Noise: An average absolute gain of 6.4% was observed. This improvement was notable even at 0 dB SNR, indicating effective noise suppression.
  • Artificial Reverberation: An average absolute gain of 3.6% was achieved.
  • Robustness Across Diverse Conditions: The proposed features demonstrated robustness against various types of additive noise (babble, jet, factory, tank, car) and different levels of reverberation.

Limitations and Future Work

  • Limitations: The paper acknowledges that the optimization procedure for the STRFs is computationally intensive. While the derived filter shows promise, its precise relationship to specific linguistic information requires further investigation.
  • Future Work: The authors suggest further research into:
  • Elucidating the relationship between the objective function, constraints, emergent STRF modulation spectra, and the speech modulation transfer function.
  • Exploring the potential of this approach for other speech processing tasks beyond phoneme recognition.
  • Investigating the role of different timescales (controlled by $\Delta T$) in shaping the emergent receptive fields and their impact on robustness.

Summary updated

BibTeX
@inproceedings{carlin2012robust,
  author = {Michael A. Carlin and Kailash Patil and Sridhar Krishna Nemala and Mounya Elhilali},
  title = {{Robust phoneme recognition based on biomimetic speech contours}},
  booktitle = {Interspeech},
  year = {2012},
}
Figure