TL;DR
This paper introduces Gaussian Flow Bridges (GFBs) for audio domain transfer, a method that modifies audio characteristics (like reverberation or distortion) without requiring paired "before and after" training data. By using a two-step deterministic flow process and a novel chunk-based minibatch Optimal Transport strategy, the authors achieve competitive results in speech restoration while preserving the original content's integrity.
What Problem Does This Paper Address?
Most state-of-the-art audio enhancement models rely on supervised learning, which requires massive datasets of paired samples (e.g., a clean recording and its exact reverberant counterpart). In the real world, obtaining these pairs is often expensive, difficult, or impossible.
While unsupervised methods like GANs or diffusion models exist, they face a major hurdle in audio: content consistency. When you ask a model to "remove reverb" without a ground-truth reference, it often hallucinates, changing the speaker's identity or making the speech unintelligible. This paper aims to bridge that gap, providing a way to perform domain transfer that is both unsupervised and content-preserving.
Methods
The core of the approach is the Gaussian Flow Bridge (GFB). Instead of trying to map one audio domain directly to another, the model uses a Gaussian distribution as an intermediate "bridge."
- Encoding: An unconditional vector field transforms the input audio ($x_0$) into a latent vector in a Gaussian noise space ($x_1$).
- Decoding: A conditional vector field then transforms that noise back into a modified audio sample ($\tilde{x}_0^c$), guided by a continuous control variable $c$ (such as a specific reverberation time $T_{60}$).
The "Straight Path" Innovation
A key challenge in flow-based models is that training trajectories can become curved and tangled, leading to artifacts. To fix this, the authors employ Flow Matching with linear interpolation. However, standard Optimal Transport (OT) couplings—used to keep these paths straight—don't scale well to high-dimensional audio waveforms.
The authors propose chunk-based minibatch OT. They partition long audio signals into small chunks (e.g., 32ms), solve the OT problem on these chunks to find the most efficient paths to noise, and then reshape them. This keeps the "trajectories" straight enough to preserve speech content without exploding computational costs.
Datasets
- Training: Studio-quality speech from the VCTK corpus.
- Reverberation: Speech was convolved with room impulse responses (RIRs) from multiple public datasets (including OpenAir and ACE), covering $T_{60}$ values from 0 to 1s.
- Evaluation: The DAPS dataset (different from training) was used, convolved with real-world RIRs not seen during training to test generalization.
- Distortion: Speech was artificially clipped at various Signal-to-Distortion Ratio (SDR) levels.
Key Insights and Ideas
- Curvature is the Enemy: The authors demonstrate that "tangled" paths during training are a primary cause of identity shifts and speech artifacts. By using chunk-based OT, they significantly reduce trajectory curvature.
- Linearity Preserves Content: Maintaining linear trajectories between data and noise is crucial for ensuring that the "content" (what is said) remains orthogonal to the "style" (the acoustic environment) being manipulated.
- The Bridge Advantage: Using a Gaussian bridge allows for many-to-many mappings. You can take any reverberant signal and "bridge" it to a specific target $T_{60}$ or $C_{50}$ value just by changing the conditioning variable during the decoding step.
Key Results
The GFB method was tested on two main tasks: dereverberation and declipping.
- Dereverberation: The method performed competitively with supervised baselines like CRUSE and STORM. While it didn't strictly beat them, the fact that it achieved similar performance without paired data is a significant win.
- Declipping: In speech declipping, the GFB significantly outperformed the popular SPADE baseline in terms of perceptual quality (NISQA scores).
- Consistency: The use of chunk-based OT (C-OT) was shown to be critical. As shown in the objective evaluations, reducing the chunk size ($N_c$) generally improved speech consistency metrics like Word Error Rate (WER) and Speaker Recognition Cosine Similarity (SR-CS).
Limitations and Future Work
While promising, the method isn't perfect. The authors note occasional inconsistencies where the model might still slightly alter the speaker's identity or misinterpret a word. There is a clear trade-off: increasing the "guidance" to get more accurate acoustic target results often comes at the cost of introducing speech artifacts.
Future work could involve refining the chunk-based OT strategy or exploring more complex backbone architectures to further improve the robustness of the content preservation. The authors also suggest that this framework could be extended to other audio tasks, such as instrument timbre transfer or environmental sound synthesis.
Summary updated