Abstract
Generative models for speech enhancement (SE) have shown great promise in robustness but suffer from high computational latency due to iterative sampling. We present SE-Bridge, a novel framework that combines Consistency Models with a Brownian Bridge stochastic process. By enforcing a consistency property along the Probability Flow ODE of a Brownian bridge, SE-Bridge enables high-fidelity speech reconstruction in a single inference step. Our approach achieves a 15x speedup over diffusion baselines and sets new state-of-the-art benchmarks in speech quality and intelligibility.
Introduction
Speech enhancement (SE) aims to isolate clean speech from noisy backgrounds, a critical task for human communication and machine perception. While discriminative models (DEMs) offer fast inference, they often over-suppress audio, leading to speech distortion. Conversely, diffusion-based generative models (GEMs) provide superior generalization to unseen noise but require hundreds of denoising steps, making them unsuitable for real-time applications.
SE-Bridge addresses this trade-off by providing a generative model that is as fast as a discriminative one but as robust as a diffusion one. It achieves this by mapping noisy input directly to clean output in a single step, maintaining the high quality associated with generative approaches.
Methodology
The SE-Bridge framework is built upon two mathematical foundations: the Brownian Bridge and Consistency Models.
Brownian Bridge Process
Unlike standard diffusion models that transition clean speech into Gaussian noise, SE-Bridge utilizes a Brownian Bridge. This process defines a stochastic path with fixed endpoints: the distribution of clean speech ($X_0$) and the distribution of noisy speech ($y$). This formulation eliminates the need to sample from a random noise prior; inference begins directly from the observed noisy signal.
Consistency Mapping
To achieve single-step inference, the model employs Consistency Models. The network is trained to enforce a "consistency" property: any point along the trajectory of the Probability Flow Ordinary Differential Equation (PF-ODE) must map back to the same initial clean state.
By learning this mapping, SE-Bridge can "jump" from the noisy input directly to the clean output in one step. Notably, SE-Bridge is trained from scratch as a standalone model, avoiding the need for distillation from a pre-trained diffusion model.
Experimental Results
The model was evaluated on standard benchmarks including WSJ0-CHiME3 and VoiceBank-DEMAND, as well as custom sets for downstream tasks.
Performance and Speed
SE-Bridge demonstrates significant improvements in both quality and efficiency. It achieved a 15x reduction in Real-Time Factor (RTF) compared to the diffusion-based SGMSE+ baseline. On the WSJ0-CHiME3 dataset, SE-Bridge reached a Scale-Invariant Signal-to-Distortion Ratio (SI-SDR) of 20.3 dB, outperforming both generative and discriminative baselines.
Downstream Task Synergy
Beyond perceptual metrics, SE-Bridge significantly enhances performance in downstream applications. In Automatic Speech Recognition (ASR) tests, it reduced the Word Error Rate (WER) to 9.62%, compared to 10.34% for diffusion baselines and 14.96% for unprocessed noisy speech. This suggests that the model preserves critical linguistic and speaker-specific information better than traditional enhancement methods.
Conclusion
SE-Bridge represents a significant step forward in efficient generative speech enhancement. By leveraging the Brownian Bridge and consistency mapping, it provides high-quality, single-step enhancement without the instability of adversarial training. Future work will explore improved phase recovery and the application of the "Bridge" framework to other signal processing domains such as image denoising and audio super-resolution.
Summary updated