A Better Critic Matters for Distribution Matching Distillation
Lists
Only you see this
Abstract
Distribution Matching Distillation (DMD) trains a few-step student using the difference between a teacher's score and an online critic's estimate of the student distribution's score. We investigate what the critic needs, in capacity and in information, to provide a reliable training signal. On the capacity side, controlled experiments across seven SiT model sizes on ImageNet show that increasing teacher size while keeping a small critic fixed can degrade student quality and cause training collapse. Scaling the critic alongside the teacher, a strategy we call joint scaling, prevents collapse in our experiments and allows the student to benefit from larger teachers. On the information side, rather than forcing the critic to model a step-agnostic mixture of student outputs, we introduce a step-aware critic (STAC) that conditions on the student's generation step to estimate step-specific scores. Together, joint scaling and STAC consistently improve few-step distillation across class-conditional ImageNet generation, text-to-image generation with SDXL and FLUX.2, and text-to-video generation with Wan2.1.
BibTeX
@misc{wu2026better,
author = {Zongze Wu and Xianbang Wang and Zhengyang Geng and Mingdeng Cao and Zhoutong Zhang and Richard Zhang and Eli Shechtman and Ruiming Cao},
title = {{A Better Critic Matters for Distribution Matching Distillation}},
year = {2026},
eprint = {2610.42135},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.42135},
}Loading related content...