TL;DR
This paper introduces an improved speech enhancement method that leverages a Multi-Band Excitation (MBE) model combined with codebook-based techniques. By refining the analysis-with-synthesis framework and incorporating speech presence probability, the method effectively removes noise not only between harmonic components but also in silent segments, leading to cleaner speech.
What Problem Does This Paper Address?
Speech enhancement aims to remove unwanted noise from audio signals, a crucial task for improving communication clarity in various applications. While traditional methods like spectral subtraction and Wiener filtering work well for stationary noise, their performance degrades significantly with non-stationary noise. Codebook-based methods offer better performance in non-stationary conditions by storing prior speech and noise spectral information. However, they often leave residual noise between harmonics and in silent segments due to limitations in modeling spectral envelopes. This paper tackles the challenge of further reducing these residual noises, particularly between harmonics and in silent regions, to achieve higher quality enhanced speech.
Methods
The proposed method builds upon the analysis-with-synthesis (AwS) framework, integrating a Multi-Band Excitation (MBE) model with codebook-based techniques.
- Codebook-Based Enhancement: The process begins with a codebook-based method to obtain an initial enhanced spectrum. This involves training linear prediction (LP) parameter codebooks for speech and noise offline.
- MBE Framework: The core of the enhancement lies in the MBE model, which is well-suited for synthesizing high-quality speech by distinguishing between voiced and unvoiced segments.
- Speech Analysis: In the analysis stage, acoustic features are extracted from the codebook-enhanced speech. These include:
- Pitch and Harmonic Magnitude: These are crucial for reconstructing the periodic structure of voiced speech.
- Voiced/Unvoiced (V/UV) Decision: This is determined by analyzing the difference between the enhanced spectrum and a reconstructed excitation spectrum, allowing for distinct synthesis strategies for voiced and unvoiced portions.
- Speech Synthesis: In the synthesis stage, voiced and unvoiced speech are generated separately. Voiced speech is synthesized using sinusoidal oscillators driven by the estimated harmonic magnitudes, frequencies, and phases. Unvoiced speech is generated by modifying random noise based on the unvoiced spectral components.
- Speech Presence Probability: A key improvement is the incorporation of speech presence probability to refine the codebook-based Wiener filter. This helps in more accurately estimating acoustic parameters by considering the likelihood of speech being present in specific frequency bins, thereby further reducing noise, especially in silent segments.
Datasets
The paper uses the NTT database for testing. The speech is corrupted with babble, white, f16, and factory2 noises from the NOISEX-92 database at various Signal-to-Noise Ratios (SNRs) (0dB, 5dB, and 10dB). The database is down-sampled to 8 kHz.
Key Insights and Ideas
- Synergy of MBE and Codebooks: Combining the harmonic modeling capabilities of MBE with the spectral envelope estimation of codebook-based methods proves effective in tackling residual noise between harmonics.
- Importance of V/UV Decision: Accurately distinguishing between voiced and unvoiced segments is critical for applying appropriate synthesis strategies and achieving cleaner speech.
- Speech Presence Probability for Silent Segments: Integrating speech presence probability significantly improves noise reduction in silent regions, a common challenge for many enhancement algorithms.
- AwS for Harmonic Reconstruction: The analysis-with-synthesis framework is well-suited for reconstructing clean harmonic structures, which is a key advantage over methods that only focus on spectral envelope.
Key Results
The proposed method demonstrates superior performance compared to two traditional codebook-based methods (Ref. A and Ref. B) across various objective metrics:
- Perceptual Evaluation of Speech Quality (PESQ): The proposed method achieved higher PESQ scores, particularly in low SNR conditions (e.g., 2.22 at 0dB, 2.50 at 5dB, 2.71 at 10dB), indicating better perceived speech quality.
- Segmental SNR (SSNR) Improvement: The method provided a greater SSNR improvement (e.g., 14.61 dB at 0dB, 12.77 dB at 5dB, 10.58 dB at 10dB), suggesting more effective noise reduction.
- Log-Spectrum Distortion (LSD): The proposed method resulted in lower LSD (e.g., 9.01 at 0dB, 7.60 at 5dB, 6.35 at 10dB), indicating less distortion in the reconstructed speech spectrum.
- Qualitative Improvement: Spectrogram analysis confirmed that the proposed method effectively removes noise between harmonics and in silent segments, producing cleaner harmonic structures.
Limitations and Future Work
- Harmonic Band Division: The paper notes that dividing frequency bands for harmonics requires a compromise between precise judgment and continuity. The best results were achieved with harmonic bands containing five harmonics.
- Future Work: Not explicitly detailed in the provided text, but the success in low SNR conditions and improved handling of silent segments suggests potential for further refinement in extremely noisy environments or for specific types of non-stationary noise. The authors' focus on the MBE model also hints at potential extensions using more advanced MBE variations.
Summary updated