TL;DR
This paper proposes a novel feature extraction method for Automatic Speech Recognition (ASR) inspired by the human auditory system's cochlear nucleus. By mimicking the parallel processing of auditory information in the cochlear nucleus, their method achieves improved recognition rates in noisy conditions compared to existing auditory models and standard MFCC features.
What Problem Does This Paper Address?
Despite significant advancements in ASR, performance still degrades rapidly in challenging conditions like speaker variability, channel mismatch, and noisy environments. Human speech perception, however, is remarkably robust to these issues. This paper aims to bridge this gap by adopting the processing strategies of the human auditory system, specifically focusing on how the cochlear nucleus (CN) processes auditory signals to enhance speech features, making them more resilient to noise and variability. Existing ASR systems often treat speech features as a single representation, whereas the CN processes different aspects of sound (like timing and intensity) in parallel pathways before integrating them. This paper argues that separating these processing streams, as observed in the CN, can lead to more robust ASR.
Methods
The proposed method models the three main divisions of the cochlear nucleus: the Anteroventral Cochlear Nucleus (AVCN), the Ventroposterior Cochlear Nucleus (PVCN), and the Dorsal Cochlear Nucleus (DCN).
- AVCN (Synchrony Detection): This pathway mimics the "primary-like" neurons of the AVCN, which preserve the timing information of the auditory nerve. The authors use a Zero-Crossing with Peak Amplitudes (ZCPA) auditory model. This model extracts features based on the timing of zero-crossings and peak amplitudes in the signal, effectively capturing the low-frequency formant information and acting as a synchrony detector.
- PVCN (Mean Rate Processing): This pathway models the "onset" and "chopper" neurons of the PVCN, which are responsible for encoding the average discharge rate of auditory nerve fibers. This is achieved using a model of the inner hair cell (IHC) synapse, which includes half-wave rectification, synaptic adaptation, and a low-pass filter to extract the signal envelope. This captures the dynamic nature of speech.
- DCN (Lateral Suppression): This pathway mimics the inhibitory effects of the DCN, similar to lateral inhibition, which helps to enhance spectral contrast. The authors implement this using a temporal companding strategy (compression and expansion) inspired by two-tone suppression observed in the DCN.
These three parallel processing streams generate separate feature sets (cepstra). These are then fused by early integration into a single, composite feature vector. The authors emphasize separating synchrony detection from synaptic processing, unlike some previous models.
Datasets
The paper uses the TIdigits dataset for isolated digit recognition. The training set consists of 55 male speakers (1210 utterances), and the test set includes 31 male speakers (682 utterances) for speaker-independent evaluation. The test speech was corrupted with both stationary Gaussian white noise and non-stationary factory noise at various Signal-to-Noise Ratios (SNRs).
Key Insights and Ideas
- Mimicking Auditory Parallelism: The core idea is that by separating and then integrating features derived from distinct processing pathways in the cochlear nucleus (synchrony vs. mean rate vs. suppression), ASR can become more robust.
- ZCPA for Synchrony: The ZCPA model effectively captures timing information crucial for speech perception, especially at lower frequencies.
- Temporal Companding for Suppression: Using temporal companding for lateral inhibition is a novel approach to spectral enhancement, inspired by the DCN's function.
- Separation of Synchrony and Mean Rate: Explicitly separating these two types of information, rather than integrating them early as in some previous models, is highlighted as a key advantage, aligning with the CN's biological processing.
- Time Constant Impact: The study suggests that a higher time constant in the low-pass filter used for envelope detection can improve performance in both clean and noisy conditions.
Key Results
The proposed feature extraction method, termed ZCPA_AUD, was evaluated using Hidden Markov Models (HMMs) on the TIdigits dataset.
- In White Noise: ZCPA_AUD performed better than the base ZCPA model in clean conditions. However, it showed some degradation in white noise at lower SNRs compared to the base ZCPA, attributed to the mean rate component's susceptibility to noise.
- In Non-Stationary Factory Noise: ZCPA_AUD demonstrated significant improvements over the base ZCPA across all tested SNRs.
- Comparison with MFCC: When compared to Mel-Frequency Cepstral Coefficients (MFCC) with delta features (MFCC_DEL), ZCPA_AUD performed equally or better than MFCC_DEL in white noise. In non-stationary factory noise, ZCPA_AUD also showed superior performance. For instance, at 5 dB SNR in factory noise, ZCPA_AUD achieved a recognition rate of 50.0%, while MFCC_DEL achieved 22.7%.
Limitations and Future Work
- Increased Dimensionality: The proposed method results in a higher dimensionality of extracted features compared to MFCC or Perceptual Linear Prediction (PLP), which can increase computational cost.
- Computational Cost: The overall approach is computationally more intensive than standard MFCC or PLP.
- Future Directions: The authors suggest optimizing the processing algorithms, particularly for the mean rate component, using noise-compensated models. They also propose exploring late integration of features with reliability factors as a future research direction.
Summary updated