Abstract
This research explores the intersection of auditory prosthetics and machine learning by investigating the performance of standard Automatic Speech Recognition (ASR) systems on "spectrally reduced speech" (SRS). SRS is a digital proxy for the signal received by cochlear implant (CI) users, where fine spectral details are removed, leaving only coarse temporal envelopes. The study demonstrates that HMM-based ASR systems trained on clean speech can achieve near-perfect accuracy on SRS, provided the signal retains at least 16 frequency subbands. Notably, the bandwidth of temporal envelopes has minimal impact on recognition accuracy, suggesting spectral resolution is the primary factor for robust ASR in these conditions.
Introduction
Human listeners with normal hearing possess a remarkable ability to understand speech even under severe degradation. In contrast, cochlear implant users rely on a highly simplified signal. To understand the essential information required for speech recognition, researchers utilize SRS—a "sinewave replica" of speech. This paper addresses whether standard ASR features, such as Mel-Frequency Cepstral Coefficients (MFCCs) and Perceptual Linear Prediction (PLP) coefficients, are robust enough to handle the specific spectral distortions inherent in CI-like speech. If an ASR system trained on high-quality speech can successfully process SRS, it indicates that current models effectively capture the core acoustic cues utilized by the human auditory system.
Methodology
The researchers employed a multi-step experimental pipeline:
- SRS Synthesis: Original speech was decomposed into $N$ subbands (ranging from 4 to 32) using a filterbank spaced on the Bark scale to mimic human auditory processing. The amplitude envelope (AM) of each band was extracted and used to modulate a sine wave at the band's center frequency.
- Variable Parameters: Two primary variables were tested:
- Spectral Resolution: The number of frequency subbands ($N = 4, 8, 16, 24, 32$).
- Temporal Detail: The cutoff frequency of the low-pass filter used for envelope extraction ($16, 50, 160, 500$ Hz).
- ASR Architecture: A speaker-independent HMM system was constructed using the HTK toolkit. Models were trained on original clean speech and evaluated on the synthesized SRS.
- Distortion Analysis: The Weighted Slope Metric (WSM) was utilized to quantify the spectral distance between the original and reduced speech, providing a mathematical baseline for distortion levels.
The study utilized the TI-digits database, a standard benchmark for connected digit sequences. The training set included over 25,000 sequences from more than 300 diverse speakers, while testing was performed on a representative subset of 250 utterances.
Results
The baseline accuracy on original clean speech was exceptionally high: 99.76% for MFCCs and 99.70% for PLPs. When testing on SRS, a significant performance "cliff" was observed. At low resolutions (4 or 8 subbands), accuracy plummeted to approximately 35-40%. However, once the resolution reached 16 subbands, accuracy improved dramatically, reaching levels nearly identical to full-spectrum speech.
As shown in the table above, at 16 subbands, MFCC accuracy reached between 96% and 99% depending on the AM bandwidth. At 32 subbands, performance was virtually indistinguishable from the baseline (~99.6%).
Discussion: The 16-Band Threshold and Temporal Envelopes
The data suggests a "tipping point" at 16 subbands where sufficient spectral information is present for the HMM to match patterns effectively. This is further supported by the distortion analysis.
The WSM analysis (shown above) confirmed that spectral distortion decreases sharply as the number of subbands increases, correlating perfectly with the observed rise in ASR accuracy. Interestingly, while temporal envelopes are crucial in human psychoacoustics, they appeared secondary for these ASR models. Changing the AM bandwidth from 500 Hz down to 16 Hz had no statistically significant effect on word accuracy. This resilience suggests that standard cepstral features like MFCCs and PLPs are capable of capturing underlying vocal tract shapes even in the absence of fine harmonic structure.
Conclusion
This study highlights the robustness of HMM-based ASR features to the spectral reductions characteristic of cochlear implants. While the "16-band rule" is established here for a small-vocabulary task like TI-digits, future work is needed to determine if this threshold holds for more complex, continuous speech. Additionally, investigating the impact of background noise on SRS recognition remains a critical area for future research, as CI users face significant challenges in noisy environments. These findings suggest that bio-inspired features, which intentionally discard irrelevant spectral information, could lead to more robust ASR systems.
Summary updated