Abstract
Target speaker extraction (TSE) aims to isolate a desired speaker's voice from a multi-speaker mixture using auxiliary information such as a reference utterance. Although recent advances in diffusion and flow-matching models have improved TSE performance, these methods typically require multi-step sampling, which limits their practicality in low-latency settings. In this work, we propose MeanFlow-TSE, a one-step generative TSE framework trained with mean-flow objectives, enabling fast and high-quality generation without iterative refinement. Building on the AD-FlowTSE paradigm, our method defines a flow between the background and target source that is governed by the mixing ratio (MR). Experiments on the Libri2Mix corpus show that our approach outperforms existing diffusion- and flow-matching-based TSE models in separation quality and perceptual metrics while requiring only a single inference step. These results demonstrate that mean-flow-guided one-step generation offers an effective and efficient alternative for real-time target speaker extraction.
Introduction
Target Speaker Extraction (TSE) is the "cocktail party" task: isolating one specific voice from a noisy, multi-speaker environment using a reference clip of that speaker.
Currently, the field is split between two approaches. Discriminative models (like Conv-TasNet) are fast and efficient but often produce "hollow" sounding audio with artificial-sounding artifacts. Generative models (like Diffusion or Flow Matching) produce much more natural, high-fidelity speech but are computationally expensive. They typically require 10 to 50 iterative steps to "refine" the audio, making them too slow for real-time applications like hearing aids or live telecommunications.
MeanFlow-TSE bridges this gap by achieving the high perceptual quality of a generative model with the one-step speed of a discriminative one.
Methodology
The researchers built their work on the AD-FlowTSE paradigm, which treats the extraction process as a "flow" between a background noise source and the target speaker.
Key technical innovations include:
- Mean-Flow Objectives: Standard flow matching models the instantaneous velocity of a point moving along a trajectory. To get to the end, you have to integrate that velocity over many small steps. MeanFlow instead models the average velocity required to jump from the starting point (the mixture) directly to the destination (the clean target) in one go.
- $\alpha$-Flow Framework: They use a hybrid target velocity that interpolates between the ground-truth trajectory and a "self-consistency" target. This helps the model learn to be accurate even when jumping across large time gaps.
- Alpha Curriculum Scheduling: The model isn't forced to learn the "one-step jump" immediately. Training starts with standard trajectory matching ($\alpha=1$) and gradually transitions to the mean-flow identity ($\alpha \to 0$) over 2,000 epochs.
- Mixing Ratio (MR) Predictor: Since the "distance" between the mixture and the clean speech depends on how loud the background noise is, the system uses a small auxiliary network (ECAPA-TDNN) to estimate the mixing ratio, which informs the starting point of the flow.
Experimental Evaluation
The model was evaluated on the Libri2Mix corpus, a standard benchmark for speech separation, using both "Clean" (two speakers) and "Noisy" (two speakers + environmental noise) versions. The setup utilized 16 kHz audio with 3-second enrollment clips to identify the target speaker within 3-second mixtures.
Key Results
MeanFlow-TSE set new state-of-the-art marks for generative TSE models on Libri2Mix, as shown in the comparison below:
- Libri2Mix Clean: Achieved an SI-SDR of 18.80 dB and a PESQ of 3.26. This is a significant jump over its predecessor, AD-FlowTSE (17.49 dB / 2.89 PESQ), despite using the same backbone architecture.
- Libri2Mix Noisy: Achieved 12.85 dB SI-SDR, outperforming complex multi-step models like FlowTSE and SoloSpeech.
- Efficiency: The model manages to outperform multi-step diffusion models in signal fidelity while maintaining a Real-Time Factor (RTF) of 0.018—meaning it can process one second of audio in just 18 milliseconds.
Discussion and Conclusion
A surprising insight from the research was that performance peaked at NFE=1 (Number of Function Evaluations). Adding more steps actually decreased performance slightly, suggesting the mean-flow objective successfully "rectifies" the flow into a straight line, where a single jump is more accurate than a multi-step path that might accumulate discretization errors.
While the results on Libri2Mix are impressive, the authors note that the current model was tested on "dry" speech. Future work will extend the MeanFlow framework to handle more complex scenarios involving reverberation and multi-channel audio. MeanFlow-TSE demonstrates that mean-flow-guided one-step generation offers an effective and efficient alternative for real-time target speaker extraction.
Summary updated