Noise-Robust Bandwidth Expansion for 8K Speech Recordings

Yin-Tse Lin, Bo-Hao Su, Chi-Han Lin, Shih-Chan Kuo, Jyh‐Shing Roger Jang, Chi-Chun Lee2023 · DOI

TL;DR

This paper introduces EP-WUN, a novel noise-robust bandwidth expansion (BWE) model for 8kHz speech that significantly improves ASR performance in noisy call center environments. By incorporating a speech quality classifier and a modified triplet loss, EP-WUN cleans speech representations while expanding bandwidth, achieving state-of-the-art results with fewer parameters.

What Problem Does This Paper Address?

Speech recordings from call centers are often narrowband (8kHz) and heavily contaminated with various noises. This poses a significant challenge for Automated Speech Recognition (ASR) systems, which typically perform best on high-sampling rate (16kHz or higher) and clean speech. The performance gap between narrowband, noisy speech and clean, wideband speech can hinder the effectiveness of call center analytics, agent training, and customer service improvements. While bandwidth expansion (BWE) techniques aim to reconstruct high-frequency components from narrowband audio, most existing methods struggle to maintain performance in the presence of real-world noise. This paper tackles the problem of simultaneously enhancing speech quality and expanding bandwidth for noisy 8kHz recordings to improve downstream ASR performance.

Methods

The core of the proposed solution is the Embedding-Polished Wave-U-Net (EP-WUN). This model builds upon the Wave-U-Net (WUN) architecture, a well-established wav-to-wav model for speech tasks. The novelty lies in two key additions:

  1. Speech Quality Classifier (SQC): A separate classifier is trained to distinguish between clean and noisy 8kHz speech embeddings. This SQC acts as a guide, helping to direct the learned representations towards a cleaner domain.
  2. Modified Triplet Loss (Lm-trp): This loss function is applied to the hidden embeddings generated by the WUN's downsampling module. It forces the "anchor" embedding (derived from the potentially noisy input) to be closer to a "clean" embedding and further away from a "noisy" embedding. This "representation cleaning" mechanism is crucial for noise robustness.

The training process involves three stages: pre-training the WUN, pre-training the SQC, and finally, end-to-end training of EP-WUN with the modified triplet loss. During evaluation, the trained EP-WUN model takes 8kHz noisy speech and outputs a reconstructed 16kHz signal, which is then fed into a standard 16kHz ASR system.

Datasets

  • Valentini-Botinhao Corpus: Used for training and evaluating the BWE model on English speech. This dataset contains 48kHz audio that is downsampled to 16kHz for targets and 8kHz for inputs, with various noisy conditions.
  • Formosa Speech in the Wild (FSW): A large-scale Taiwanese Mandarin corpus collected from broadcast radio. Used for training the BWE model for Mandarin speech.
  • E.SUN-IVR-cust: A newly introduced, proprietary real-world Mandarin Chinese dataset collected from an interactive voice response (IVR) system in a call center. This dataset consists of 8kHz recordings and serves as a challenging, unseen test set for ASR evaluation.

Key Insights and Ideas

  • Representation Cleaning is Key for Noise Robustness: The paper highlights that simply expanding bandwidth isn't enough for noisy speech. Actively cleaning the intermediate speech representations using a quality classifier and triplet loss is essential for robust BWE.
  • Speech Quality Metrics Don't Always Correlate with ASR Performance: While traditional speech quality metrics like PESQ and STOI are important, the authors observe that a lower Log-Spectral Distance (LSD) metric appears to be a better indicator of improved ASR performance, suggesting that spectral reconstruction accuracy is paramount for downstream ASR.
  • Smaller Models Can Be More Effective: EP-WUN achieves competitive or superior results to larger, state-of-the-art models while using significantly fewer parameters (up to 80% fewer than UEE and 33% fewer than MTL-MBE). This is crucial for practical deployment in resource-constrained environments.
  • Real-World Evaluation is Critical: Testing on a proprietary, in-the-wild call center dataset (E.SUN-IVR-cust) demonstrates the model's practical applicability and robustness beyond controlled laboratory conditions.

Key Results

  • Speech Quality Metrics: EP-WUN achieves competitive or better performance on several speech quality metrics (PESQ, STOI, LSD, CSIG, CBAK, COVL) compared to state-of-the-art models like MTL-MBE, particularly excelling in metrics sensitive to noise (CSIG, COVL) and spectral reconstruction (LSD).
  • ASR Performance:
    • For English, EP-WUN achieved a 10% reduction in Word Error Rate (WER) compared to MTL-MBE (28.77% vs. 38.63%).
  • For Mandarin Chinese, EP-WUN achieved a 11.71% reduction in WER compared to MTL-MBE (24.98% vs. 36.69%) on the unseen E.SUN-IVR-cust dataset.
  • Parameter Efficiency: EP-WUN uses 33% fewer parameters than the MTL-MBE model.

Limitations and Future Work

  • Additive Noise Assumption: The current work primarily addresses additive noise. The authors acknowledge that real-world call center audio can involve more complex distortions like reverberation, clipping, and room impulse responses.
  • Future Directions: The authors plan to extend their work to handle these more complex distortions to further improve the generalizability of their model. They also aim to conduct more detailed experiments to confirm their hypothesis about the importance of spectral-related metrics for ASR performance.

Summary updated

BibTeX
@misc{lin2023noise,
  author = {Yin-Tse Lin and Bo-Hao Su and Chi-Han Lin and Shih-Chan Kuo and Jyh‐Shing Roger Jang and Chi-Chun Lee},
  title = {{Noise-Robust Bandwidth Expansion for 8K Speech Recordings}},
  year = {2023},
  doi = {10.21437/interspeech.2023-857},
}
Figure