Abstract
Modern monaural speech enhancement often relies on "direct-fitting" neural networks that map noisy inputs to clean targets. However, these models frequently suffer from over-suppression or residual noise because they fail to explicitly account for the intrinsic acoustic characteristics of human speech. This paper introduces the Refining Underlying Information (RUI) framework, which treats speech enhancement as an iterative information-gathering process based on the Information Bottleneck (IB) principle. By capturing underlying articulatory attributes and harmonic structures, RUI achieves a significant 0.3+ boost in PESQ scores with minimal computational overhead.
Introduction
Most contemporary speech enhancement systems utilize deep neural networks to perform direct mapping from a noisy spectrum to a clean target. While effective, these models often struggle in complex acoustic environments, leading to the loss of natural speech components (over-suppression) or the retention of unwanted artifacts (under-suppression). The core issue is that these models do not explicitly model the intrinsic acoustic characteristics—such as articulatory structures and harmonics—that make speech sound natural to human ears. The RUI framework addresses this by theoretically and practically incorporating these "hidden" speech properties to repair degraded audio.
Methodology
The RUI framework is designed as an incremental refinement process grounded in the Information Bottleneck (IB) principle. It moves beyond simple spectral mapping to target the "missing" underlying information in speech signals.
Framework Architecture
The framework consists of three primary components:
- Pre-enhancement Module (PEM): A baseline model (e.g., DPCRN or CRN) that generates an initial, rough estimate of the clean speech.
- Underlying Information Extractor (UIE): Utilizing a "Harmonic Attention" mechanism, this module captures the comb-like harmonic structures inherent in human speech that the PEM might overlook.
- Multiple Refinement Iterator (MRI): The core innovation of the framework. Based on the chain rule of entropy, the MRI employs a dual-path residual mechanism—an S-path for spectral features and an A-path for acoustic features—to iteratively "fill in the gaps" of the underlying information.
To ensure psychoacoustic fidelity, the framework is optimized using a composite loss function ($L_{AC}$) that combines SI-SNR for signal fidelity and PMSQE for perception-based quality.
Experimental Setup
The framework was rigorously evaluated using the 2020 DNS-Challenge (Deep Noise Suppression) dataset. The training set comprised 500 hours of clean speech and over 180 hours of noise. Final comparisons were conducted on 300 hours of noisy speech with Signal-to-Noise Ratios (SNRs) ranging from -5 dB to 20 dB.
Results and Discussion
The RUI framework demonstrated superior performance across multiple benchmarks:
- Quality Improvements: RUI consistently improved PESQ scores by over 0.3 compared to baseline models.
- Efficiency: These gains were achieved with the addition of only 1.18 million parameters. At a total of 2.88M parameters, RUI outperformed significantly larger models like FullSubNet+ (8.67M) and GaGNet (5.94M) in SI-SDR and STOI metrics.
- Qualitative Analysis: Spectrogram analysis confirms that RUI successfully restores harmonic structures (the distinct horizontal stripes in a spectrogram) that are often blurred or deleted by standard pre-enhancement modules.
Conclusion
The RUI framework offers a flexible, "plug-and-play" solution for enhancing existing speech models. By reframing speech enhancement as a compression and refinement task under the Information Bottleneck principle, it effectively captures the underlying acoustic characteristics necessary for natural-sounding speech. Future work will explore more diverse acoustic modeling beyond harmonics to further improve hearing-repair capabilities.
Summary updated