eDMD: Endpoint Distribution Matching for Few-Step Diffusion Distillation

Tong Zhao, Xianfang Zeng, Yucheng Han, Ruoyu Wang, Mingkun Lei, Gang Yu, Chi ZhangarXiv 2026 · arXiv

TL;DR

eDMD (Endpoint Distribution Matching Distillation) significantly improves the quality of few-step diffusion models by training an auxiliary "fake" score model exclusively on the student's final outputs. By decoupling the distillation signal from intermediate denoising states, the authors eliminate a persistent mathematical bias (the "reference residual"), allowing 4-step models to achieve photorealistic results that rival their 40-step teachers.

What Problem Does This Paper Address?

Diffusion models have revolutionized image generation, but their iterative nature makes them slow, often requiring 30–50 network evaluations to produce a single high-quality image. Distillation aims to "compress" this process into just 1–4 steps.

A leading technique for this is Distribution Matching Distillation (DMD). In DMD, a "student" model is trained using two signals: a "teacher" model (the original slow model) and a "fake" model (an auxiliary model trained online to track the student's current distribution). The student is pushed toward the teacher and away from the "fake" distribution.

The authors identify a critical flaw in how multi-step DMD is currently handled. Existing methods train the "fake" model on intermediate denoising steps or a mixture of all steps. However, in a few-step student, the intermediate states don't necessarily need to follow the teacher's distribution—only the final "endpoint" image matters. Forcing intermediate steps to match a reference creates a "reference residual"—a mathematical ghost that continues to push the student away from the target even after it has technically converged.

Methods

The core of the paper is Endpoint Distribution Matching Distillation (eDMD), which introduces three key innovations:

  1. Endpoint Reference: Unlike previous methods that train the fake score model on all denoising steps, eDMD trains a single shared fake model exclusively on the student's final generated outputs. This ensures the correction signal vanishes exactly when the student's final output matches the teacher's distribution.
  2. Precise Teacher (PT): Standard distillation uses a single-step teacher prediction as a target. However, single-step predictions can be "blurry" because they average over many possible outcomes. eDMD uses a "Precise Teacher" that performs a short multi-step trajectory (e.g., 4 Euler steps) for the final denoising interval, providing a much sharper and more accurate target.
  3. Calibrated Query Density: Not all noise levels provide equally reliable supervision. The authors introduce a calibration step that estimates where the teacher model is most consistent across noise levels and concentrates training queries in those "reliable" zones.
Figure 2. Overview of eDMD on a 2D tree-structured toy distribution (gray). Left: a 4-step student denoises from noise (σ = 1) to its final output (σ = 0, “delivered”); each update noises one step’s prediction to a level τ (green bar: its range) and compares the teacher and the fake model at the resulting query y. Right: compared with DMD2 and TDM, eDMD trains the fake model on final outputs only (top), draws the queries of intermediate steps from a calibrated density (middle; ticks: the level each step denoises to), and answers low-noise queries with a multi-step teacher instead of a single prediction (bottom; percentages: answers on the tree).

Datasets

The authors evaluated eDMD across four major diffusion and flow-matching backbones:

  • SD3-Medium
  • FLUX.2-klein (4B and 9B versions)
  • Qwen-Image-2.1

For training, they used a corpus of 191,500 prompts (compiled from T2I-2M, LAION, and others) and prompt-enhanced rewrites for the Qwen models. Evaluation was performed on DrawBench-1000 and DPG-Bench to measure human preference, aesthetic quality, and prompt adherence.

Key Insights and Ideas

  • The Reference Residual: The paper's most significant theoretical contribution is proving that training on intermediate states introduces a non-zero residual. This residual can actually deflect the student away from the desired distribution.
  • Intermediate Freedom: By only constraining the endpoint, intermediate steps are given the "freedom" to take whatever form best suits a 4-step generation process. On FLUX models, for instance, the authors found that intermediate predictions actually grow to nearly 3x the norm of the final output to achieve better results—a behavior that would be suppressed by traditional distillation.
  • On-Tree vs. Off-Tree Targets: Figure 2 (bottom right) illustrates that single-step teacher calls often fall "off-tree" (between branches of the data distribution), while the Precise Teacher "walks" the teacher's ODE to stay "on-tree," leading to better fine-grained details like fur and text.
Qualitative results on FLUX.2-klein-4B. eDMD preserves richer details in fur and hair compared to baselines.

Key Results

eDMD consistently outperformed existing state-of-the-art distillation methods (like DMD2, TDM, and CDM) across all tested backbones:

  • State-of-the-Art Performance: eDMD achieved the highest HPSv3++ (Human Preference Score) across all four backbones for 4-step generation.
  • Visual Fidelity: As seen in Figure 1, the 4-step distilled Qwen-Image-2.1 model renders photorealistic scenes and legible text, which is notoriously difficult for few-step models.
  • Convergence: In synthetic 2D experiments, the endpoint reference was the only method that allowed the student to perfectly match the teacher's complex tree-like distribution without being deflected by intermediate biases.
Figure 1. 4-step samples from eDMD, distilled from Qwen-Image-2.1. Using a tenth of the teacher’s 40 steps, eDMD renders photorealistic scenes, legible text and diverse artistic styles.

Limitations and Future Work

  • Training Overhead: While eDMD doesn't increase inference cost, it does require training an auxiliary fake model online, which adds to the computational budget during the distillation phase.
  • Teacher Dependence: The quality of the student is still capped by the teacher. If the teacher has inherent biases or artifacts, the student will likely inherit them.
  • Step Count: While 4-step generation is a massive improvement over 40 steps, the industry continues to push toward 1-step "lightning" models. The authors suggest that the endpoint principle is even more critical as the step count decreases.

Future work could involve exploring how endpoint matching interacts with adversarial losses or applying these principles to video and audio distillation.

Summary updated

Abstract

Distribution matching distillation (DMD) has emerged as a leading paradigm for distilling diffusion models into few-step students, where an auxiliary fake score model tracks the student's distribution and the discrepancy between the teacher and fake predictions drives the student toward the teacher. In multi-step distillation, although only the student's final output is delivered, existing methods train the fake model on intermediate states or across all denoising steps. Consequently, the student's update signal becomes entangled with intermediate distributions, which can distort the matching direction and remain nonzero even after the student's final outputs already match the teacher distribution. To resolve this mismatch, we propose Endpoint Distribution Matching Distillation (eDMD), which trains a single shared fake model exclusively on the student's final outputs and uses it to correct every denoising step. Our theoretical analysis proves that this endpoint reference guarantees that the distribution matching term vanishes if and only if the student's final output distribution matches the teacher's, regardless of intermediate distributions. Beyond the fake reference, we further strengthen teacher supervision through a precise teacher (PT) that replaces single-step guided predictions with short multi-step trajectories on the final noise interval, paired with a calibrated query density for intermediate steps. Experiments on SD3-Medium, FLUX.2-klein (4B and 9B), and Qwen-Image-2.1 demonstrate that eDMD consistently outperforms existing distribution matching methods for 4-step generation.
BibTeX
@misc{zhao2026edmd,
  author = {Tong Zhao and Xianfang Zeng and Yucheng Han and Ruoyu Wang and Mingkun Lei and Gang Yu and Chi Zhang},
  title = {{eDMD: Endpoint Distribution Matching for Few-Step Diffusion Distillation}},
  year = {2026},
  eprint = {2611.17194},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.17194},
}
Figure