From Flow to One Step: Real-Time Multi-Modal Trajectory Policies via Implicit Maximum Likelihood Estimation-based Distribution Distillation

Ju Dong, Liding Zhang, Lei Zhang, Yu Fu, Kaixin Bai, Zoltán-Csaba Márton, Zhenshan Bing, Zhaopeng Chen, Alois Knoll, Jianwei Zhang

Abstract: Generative policies based on diffusion and flow matching achieve strong performance in robotic manipulation by modeling multi-modal human demonstrations. However, their reliance on iterative Ordinary Differential Equation (ODE) integration introduces substantial latency, limiting high-frequency closed-loop control. Recent single-step acceleration methods alleviate this overhead but often exhibit distributional collapse, producing averaged trajectories that fail to execute coherent manipulation strategies. We propose a framework that distills a Conditional Flow Matching (CFM) expert into a fast single-step student via Implicit Maximum Likelihood Estimation (IMLE). A bi-directional Chamfer distance provides a set-level objective that promotes both mode coverage and fidelity, enabling preservation of the teacher multi-modal action distribution in a single forward pass. A unified perception encoder further integrates multi-view RGB, depth, point clouds, and proprioception into a geometry-aware representation. The resulting high-frequency control supports real-time receding-horizon re-planning and improved robustness under dynamic disturbances.

Introduction

Generative models such as Diffusion and Flow Matching have revolutionized robotic manipulation by enabling robots to handle "multi-modality"—the existence of multiple valid strategies for a single task. However, these models are computationally expensive, requiring iterative denoising or flow steps that limit inference frequency to approximately 2–10 Hz. In robotics, such low frequencies hinder reactivity; if an environment changes during a calculation cycle, the robot cannot adjust quickly enough to avoid failure. Previous attempts to accelerate these models to single-step inference often result in mode collapse, where the model averages all possible strategies into a single, physically nonsensical trajectory.

Methodology: Flow to One Step

The authors introduce a teacher-student distillation framework designed to achieve one-step inference without sacrificing multi-modal diversity.

The "Flow to One Step" framework distills a slow (2-3Hz) CFM Teacher Policy into a high-frequency (100-125Hz) Student Policy using multi-modal distillation and a unified perception encoder.

Teacher-Student Distillation

  1. The Teacher: A Conditional Flow Matching (CFM) model that learns to map simple noise to complex trajectory distributions. While expressive, it is slow due to the required ODE solving at inference time.
  2. The Student: A single-step policy using a 1D U-Net architecture that maps a noise vector and observation directly to a full trajectory in one pass.
  3. IMLE Distillation: To prevent averaging, the framework uses Implicit Maximum Likelihood Estimation (IMLE). The student generates a set of "hypotheses" rather than a single point estimate.
  4. Bi-directional Chamfer Distance: This loss function ensures the student's distribution matches the teacher's. It includes a mode-covering component (ensuring all teacher strategies are represented) and a mode-seeking component (ensuring student outputs are valid strategies according to the teacher).

Unified Perception

The policy utilizes a geometry-aware encoder that fuses multi-view RGB, depth maps, point clouds, and proprioception. Cross-attention and gated fusion are employed to ensure robustness against sensor noise and dynamic disturbances.

Experimental Results

The framework was evaluated on the RLBench simulation benchmark and real-world tasks using a 7-DoF Franka Emika Panda robot.

Dynamic Pick and Place task demonstrating the student policy's ability to maintain multiple trajectory modes (1, 2, 3) while responding to dynamic environment changes.

Performance and Speed

The student policy achieved inference speeds of 123.5 Hz to 125 Hz, representing a 43× speedup over the CFM teacher. In RLBench simulation, the student achieved a 68.6% success rate, vastly outperforming the Consistency Policy baseline (16.3%) and nearly matching the 50-step teacher (74.1%).

Real-World Robustness

In dynamic tasks where objects were moved by humans during execution, the teacher and other slow baselines failed completely (0% success rate) due to latency. In contrast, the distilled student achieved 50–66.7% success by utilizing its high frequency for real-time "receding-horizon re-planning," effectively correcting its path 125 times per second.

Conclusion

By combining IMLE-based distillation with a bi-directional Chamfer loss, the "Flow to One Step" framework successfully bridges the gap between expressive generative modeling and real-time robotic control. This approach preserves the multi-modal intelligence of slow teacher models while providing the high-frequency reactivity necessary for deployment in dynamic, real-world environments.

Summary updated

Abstract

Generative policies based on diffusion and flow matching achieve strong performance in robotic manipulation by modeling multi-modal human demonstrations. However, their reliance on iterative Ordinary Differential Equation (ODE) integration introduces substantial latency, limiting high-frequency closed-loop control. Recent single-step acceleration methods alleviate this overhead but often exhibit distributional collapse, producing averaged trajectories that fail to execute coherent manipulation strategies. We propose a framework that distills a Conditional Flow Matching (CFM) expert into a fast single-step student via Implicit Maximum Likelihood Estimation (IMLE). A bi-directional Chamfer distance provides a set-level objective that promotes both mode coverage and fidelity, enabling preservation of the teacher multi-modal action distribution in a single forward pass. A unified perception encoder further integrates multi-view RGB, depth, point clouds, and proprioception into a geometry-aware representation. The resulting high-frequency control supports real-time receding-horizon re-planning and improved robustness under dynamic disturbances.
BibTeX
@misc{dong2026from,
  author = {Ju Dong and Liding Zhang and Lei Zhang and Yu Fu and Kaixin Bai and Zoltán-Csaba Márton and Zhenshan Bing and Zhaopeng Chen and Alois Knoll and Jianwei Zhang},
  title = {{From Flow to One Step: Real-Time Multi-Modal Trajectory Policies via Implicit Maximum Likelihood Estimation-based Distribution Distillation}},
  year = {2026},
  eprint = {2603.09415},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2603.09415},
}
Figure