MeanSE: Efficient Generative Speech Enhancement with Mean Flows

Jiahe Wang, Hongyu Wang, Zhao‐Jun Wei, Deqiang Wang, Chenda Li, W. Zhang, Lixing Tan, Yanmin QianarXiv 2025 · DOI · arXiv

TL;DR

MeanSE introduces "mean flows" to generative speech enhancement, allowing a model to predict the average transformation path from noisy to clean speech in a single step. By modeling the average velocity field rather than instantaneous changes, MeanSE achieves high-quality restoration with only one function evaluation (1-NFE), significantly outperforming standard flow-matching models in both efficiency and out-of-domain generalization.

What Problem Does This Paper Address?

Speech Enhancement (SE) aims to recover clean speech from signals degraded by noise and reverberation. While traditional "discriminative" models are fast, they often produce muffled or artificial-sounding results. Recently, generative models like Diffusion and Flow Matching have set new standards for perceptual quality.

However, these generative models have a major drawback: they are computationally expensive. They work by iteratively solving differential equations, requiring multiple "Number of Function Evaluations" (NFEs)—often 5 to 50 passes through a neural network—to produce a single output. This makes them unsuitable for real-time applications on edge devices. While "FlowSE" (a flow-matching baseline) reduced the NFE requirement, its performance still degrades sharply when pushed to a single-step (1-NFE) inference. MeanSE tackles this "efficiency vs. quality" trade-off head-on.

Methods

The core innovation is the shift from Instantaneous Velocity to Mean Flow.

  • The Concept: Standard flow matching (like FlowSE) learns the instantaneous velocity of a data point as it moves from a noisy distribution to a clean one. To get to the destination, you have to take many small steps along that velocity field. MeanSE instead learns to predict the average velocity over a time interval.
  • 1-NFE Inference: Because the model knows the average velocity for the entire path (from $t=1$ to $t=0$), it can jump directly to the clean estimate in one step: $x_0 = x_1 - u_\theta(x_1, 0, 1, y)$.
  • Training Stabilizers: The authors found that training a mean flow model from scratch is unstable. They introduced two key strategies:
  • Flow Field Mix-up: During training, the model occasionally samples a zero-length time interval, forcing it to also learn instantaneous velocity, which acts as a regularizer.
  • Time Interval Curriculum Learning: The model is initialized with a pre-trained FlowSE checkpoint. Training starts with very small time intervals, which are gradually increased to the full $[0, 1]$ range. This "walk before you run" approach ensures the model learns the complex average trajectories effectively.

Datasets

The researchers evaluated the model on two primary benchmarks:

  • VoiceBank-DEMAND: Used for in-domain training and testing. It consists of clean speech mixed with various environmental noises (e.g., office, cafe).
  • WHAMR!: Used to test out-of-domain generalization. This is a much more challenging dataset as it includes synthetic reverberation (echoes), which the model did not see during training on VoiceBank-DEMAND.

Key Insights and Ideas

  • The "Shortcut" Advantage: By predicting the average velocity, MeanSE essentially learns the "chord" of the curved ODE path. This allows it to bypass the discretization errors that plague standard flow models when they are forced to take large, single steps.
  • Superior Generalization: One of the most striking findings is that MeanSE generalizes to unseen conditions (like reverberation in WHAMR!) far better than standard flow matching. While FlowSE's 1-NFE performance collapses on out-of-domain data, MeanSE remains robust.
  • Generative Integrity: Unlike other 1-NFE methods that rely on "data prediction" (which can make generative models behave more like blurry discriminative ones), MeanSE maintains the characteristics of a generative flow, preserving naturalness and perceptual detail.

Key Results

  • In-Domain (1-NFE): On the VoiceBank-DEMAND test set, MeanSE achieved a PESQ score of 2.090, significantly higher than FlowSE’s 1.843.
  • Out-of-Domain (1-NFE): On the WHAMR! dataset, the gap widened further. MeanSE outperformed the baseline across all non-intrusive metrics, including a massive gain in WVMOS (+1.090) and NISQA (+0.511).
  • Efficiency: MeanSE at 1-NFE performs comparably to (or better than) FlowSE at 2 or 5 NFEs, effectively providing a 2x to 5x speedup for the same quality level.

Limitations and Future Work

The authors note that while MeanSE is highly efficient, it is currently focused on speech enhancement. They suggest that the "mean flow" framework is a general-purpose tool that could be applied to other speech-processing tasks, such as Text-to-Speech (TTS) or Voice Conversion.

One potential limitation not deeply explored is the memory overhead of the "two-time-step" embedding fusion (processing both start and end times $r$ and $t$), though this is likely negligible compared to the savings gained from reducing NFEs. The researchers plan to explore these mean flow applications in broader speech tasks in the future.

Summary updated

Abstract

Speech enhancement (SE) improves degraded speech's quality, with generative models like flow matching gaining attention for their outstanding perceptual quality. However, the flow-based model requires multiple numbers of function evaluations (NFEs) to achieve stable and satisfactory performance, leading to high computational load and poor 1-NFE performance. In this paper, we propose MeanSE, an efficient generative speech enhancement model using mean flows, which models the average velocity field to achieve high-quality 1-NFE enhancement. Experimental results demonstrate that our proposed MeanSE significantly outperforms the flow matching baseline with a single NFE, exhibiting extremely better out-of-domain generalization capabilities.
BibTeX
@misc{wang2025meanse,
  author = {Jiahe Wang and Hongyu Wang and Zhao‐Jun Wei and Deqiang Wang and Chenda Li and W. Zhang and Lixing Tan and Yanmin Qian},
  title = {{MeanSE: Efficient Generative Speech Enhancement with Mean Flows}},
  year = {2025},
  eprint = {2509.21214},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2509.21214},
  howpublished = {arXiv},
}
Figure