MeanFlow-TSE: One-Step Generative Target Speaker Extraction with Mean Flow

Riki Shimizu, Xilin Jiang, Nima MesgaraniarXiv 2025 · DOI · arXiv

Abstract

Target speaker extraction (TSE) aims to isolate a desired speaker's voice from a multi-speaker mixture using auxiliary information such as a reference utterance. Although recent advances in diffusion and flow-matching models have improved TSE performance, these methods typically require multi-step sampling, which limits their practicality in low-latency settings. In this work, we propose MeanFlow-TSE, a one-step generative TSE framework trained with mean-flow objectives, enabling fast and high-quality generation without iterative refinement. Building on the AD-FlowTSE paradigm, our method defines a flow between the background and target source that is governed by the mixing ratio (MR). Experiments on the Libri2Mix corpus show that our approach outperforms existing diffusion- and flow-matching-based TSE models in separation quality and perceptual metrics while requiring only a single inference step. These results demonstrate that mean-flow-guided one-step generation offers an effective and efficient alternative for real-time target speaker extraction.

Introduction

Target Speaker Extraction (TSE) is the "cocktail party" task: isolating one specific voice from a noisy, multi-speaker environment using a reference clip of that speaker.

Currently, the field is split between two approaches. Discriminative models (like Conv-TasNet) are fast and efficient but often produce "hollow" sounding audio with artificial-sounding artifacts. Generative models (like Diffusion or Flow Matching) produce much more natural, high-fidelity speech but are computationally expensive. They typically require 10 to 50 iterative steps to "refine" the audio, making them too slow for real-time applications like hearing aids or live telecommunications.

MeanFlow-TSE bridges this gap by achieving the high perceptual quality of a generative model with the one-step speed of a discriminative one.

Methodology

The researchers built their work on the AD-FlowTSE paradigm, which treats the extraction process as a "flow" between a background noise source and the target speaker.

Comparison of inference steps across different generative TSE paradigms: Flow Matching (many steps), AD-FlowTSE (few steps), and the proposed MeanFlow-TSE (one step).

Key technical innovations include:

  • Mean-Flow Objectives: Standard flow matching models the instantaneous velocity of a point moving along a trajectory. To get to the end, you have to integrate that velocity over many small steps. MeanFlow instead models the average velocity required to jump from the starting point (the mixture) directly to the destination (the clean target) in one go.
  • $\alpha$-Flow Framework: They use a hybrid target velocity that interpolates between the ground-truth trajectory and a "self-consistency" target. This helps the model learn to be accurate even when jumping across large time gaps.
  • Alpha Curriculum Scheduling: The model isn't forced to learn the "one-step jump" immediately. Training starts with standard trajectory matching ($\alpha=1$) and gradually transitions to the mean-flow identity ($\alpha \to 0$) over 2,000 epochs.
  • Mixing Ratio (MR) Predictor: Since the "distance" between the mixture and the clean speech depends on how loud the background noise is, the system uses a small auxiliary network (ECAPA-TDNN) to estimate the mixing ratio, which informs the starting point of the flow.

Experimental Evaluation

The model was evaluated on the Libri2Mix corpus, a standard benchmark for speech separation, using both "Clean" (two speakers) and "Noisy" (two speakers + environmental noise) versions. The setup utilized 16 kHz audio with 3-second enrollment clips to identify the target speaker within 3-second mixtures.

Key Results

MeanFlow-TSE set new state-of-the-art marks for generative TSE models on Libri2Mix, as shown in the comparison below:

Performance comparison of MeanFlow-TSE against state-of-the-art discriminative and generative models on the Libri2Mix dataset.
  • Libri2Mix Clean: Achieved an SI-SDR of 18.80 dB and a PESQ of 3.26. This is a significant jump over its predecessor, AD-FlowTSE (17.49 dB / 2.89 PESQ), despite using the same backbone architecture.
  • Libri2Mix Noisy: Achieved 12.85 dB SI-SDR, outperforming complex multi-step models like FlowTSE and SoloSpeech.
  • Efficiency: The model manages to outperform multi-step diffusion models in signal fidelity while maintaining a Real-Time Factor (RTF) of 0.018—meaning it can process one second of audio in just 18 milliseconds.

Discussion and Conclusion

A surprising insight from the research was that performance peaked at NFE=1 (Number of Function Evaluations). Adding more steps actually decreased performance slightly, suggesting the mean-flow objective successfully "rectifies" the flow into a straight line, where a single jump is more accurate than a multi-step path that might accumulate discretization errors.

While the results on Libri2Mix are impressive, the authors note that the current model was tested on "dry" speech. Future work will extend the MeanFlow framework to handle more complex scenarios involving reverberation and multi-channel audio. MeanFlow-TSE demonstrates that mean-flow-guided one-step generation offers an effective and efficient alternative for real-time target speaker extraction.

Summary updated

Abstract

Target speaker extraction (TSE) aims to isolate a desired speaker's voice from a multi-speaker mixture using auxiliary information such as a reference utterance. Although recent advances in diffusion and flow-matching models have improved TSE performance, these methods typically require multi-step sampling, which limits their practicality in low-latency settings. In this work, we propose MeanFlow-TSE, a one-step generative TSE framework trained with mean-flow objectives, enabling fast and high-quality generation without iterative refinement. Building on the AD-FlowTSE paradigm, our method defines a flow between the background and target source that is governed by the mixing ratio (MR). Experiments on the Libri2Mix corpus show that our approach outperforms existing diffusion- and flow-matching-based TSE models in separation quality and perceptual metrics while requiring only a single inference step. These results demonstrate that mean-flow-guided one-step generation offers an effective and efficient alternative for real-time target speaker extraction. Code is available at https://github.com/rikishimizu/MeanFlow-TSE.
BibTeX
@misc{shimizu2025meanflow,
  author = {Riki Shimizu and Xilin Jiang and Nima Mesgarani},
  title = {{MeanFlow-TSE: One-Step Generative Target Speaker Extraction with Mean Flow}},
  year = {2025},
  eprint = {2512.18572},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2512.18572},
}

Loading related content...

Figure