Are We Overthinking One-Step Generation?

Woobin Park, Jaeyeon Kim, Yilun Du, Sitan Chen, Bohyung HanarXiv 2026 · arXiv

TL;DR

Priming Models simplify one-step generative modeling into a straightforward two-stage process: simple reconstruction pretraining (denoising at a fixed noise level) followed by distributional post-training in feature space. This "minimalist" approach achieves state-of-the-art results on ImageNet (FID 1.19) while requiring up to 15x less training compute than previous one-step methods.

What Problem Does This Paper Address?

While diffusion and flow-matching models produce stunning images, they are notoriously slow because they require multiple network evaluations (steps) to generate a single sample. This has led to a "one-step" arms race.

However, current one-step solutions are often incredibly complex. They typically rely on "distillation" (training a student model to mimic a slow teacher), complex noise trajectories, or specialized stabilization tricks. The authors of this paper ask a provocative question: Are we overthinking this? They set out to prove that you don't need complex trajectories or teacher models to train a high-quality one-step generator from scratch; you just need a better starting point.

Methods

The authors introduce a two-stage framework called Priming Models:

  1. Stage 1: Reconstruction Pretraining: Instead of starting from random weights, the model is first trained on a simple reconstruction task. Specifically, it learns to denoise images corrupted by a fixed amount of Gaussian noise (e.g., 80% noise). This "primes" the model to understand the basic structure and manifold of natural images.
  2. Stage 2: Distributional Post-training: The pretrained model is then treated as a one-step generator that maps pure Gaussian noise to images. It is refined by matching the distribution of its outputs to the distribution of real data in a frozen feature space (using an Optimal Transport/Sinkhorn loss).

Unlike prior work, this method doesn't require classifier-free guidance, complex style-conditioning, or a pretrained teacher model.

Datasets

The researchers evaluated Priming Models across three distinct and challenging domains:

  • Class-conditional Image Generation: ImageNet-1K at $256 \times 256$ resolution.
  • Text-to-Image Generation: CC12M and curated high-quality datasets, evaluated on GenEval and DPG-Bench.
  • World Modeling (Robotics): The RT-1 and Bridge-V2 datasets, where the model predicts the next video frame based on robot actions.

Key Insights and Ideas

  • The "Aha!" Moment: Distributional training (like GANs or direct matching) is notoriously difficult to start from scratch because the network initially has zero knowledge of image structure. The authors found that a simple reconstruction task is enough to entirely alleviate this optimization hurdle.
  • Jacobian Alignment: Why does pretraining help so much? The authors show that pretraining aligns the model's Jacobian (how its outputs change with respect to its parameters) with the directions needed to correct the distribution. This makes the subsequent post-training updates significantly more efficient.
  • Fixed Noise Suffices: Surprisingly, you don't need a complex noise schedule for pretraining. Training at a single, high noise level (like $s=0.8$) provides an effective initialization that is "close enough" to the pure noise ($s=1.0$) used during generation.
Priming Models dramatically improve the training compute vs. FID trade-off (left) and show how pretraining accelerates post-training (right).

Key Results

  • State-of-the-Art Quality: On ImageNet, Priming-L achieved an FID of 1.19, the best reported for any one-step generator.
  • Massive Efficiency Gains: As shown in Figure 1, Priming Models reached SOTA performance using approximately 1/10th to 1/15th the training compute of prior methods like Drifting Models or WF-XL.
  • Versatility: In world modeling, Priming Models outperformed the one-step "DriftWorld" baseline and even a 32-step flow-matching baseline across almost all metrics, while using significantly less compute.
  • Multi-step Extension: While designed for one-step, the framework can be extended to 2 or 4 steps, further improving quality and rivaling much more expensive models.
Figure 2: One-step samples from Priming Models across three visual domains.

Limitations and Future Work

  • Reliance on Frozen Encoders: The distributional loss currently depends on pretrained feature encoders (like MAE or ConvNeXt). A major open question is whether this dependency can be eliminated to create a truly "from-scratch" system.
  • The Performance Gap: While Priming Models are the best one-step generators, there is still a small quality gap compared to the very best multi-step diffusion models.
  • Theoretical Depth: While the authors provide empirical evidence for "Jacobian alignment," a more rigorous theoretical framework for why this specific pretraining task works so well remains a direction for future research.

Summary updated

Abstract

Diffusion and flow matching are now dominant frameworks for generative modeling in continuous domains, but they typically require multiple network evaluations at inference. This has motivated extensive research on one-step generative modeling primarily built on diffusion and flow, though existing approaches often rely on complex training machinery. In this work, we introduce Priming Models, a simple two-stage framework for training one-step generators. Motivated by the difficulty of distributional training from scratch, Priming Models are first trained on a reconstruction objective, e.g., pixel-space denoising at a fixed noise level, before undergoing feature-space distributional post-training. Our central scientific finding is that this simple reconstruction stage is already enough to entirely alleviate the difficulty of distributional training. Empirically, across class-conditional image generation, text-to-image generation, and world modeling, Priming Models significantly advance the Pareto frontier of training compute vs. generation quality. On ImageNet at $256 \times 256$ resolution, we achieve a state-of-the-art FID of $1.19$ among one-step generators using approximately one-tenth the training compute of prior methods with comparable FID. We open-source our codebase at https://github.com/JaeyeonKim01/priming.
BibTeX
@misc{park2026are,
  author = {Woobin Park and Jaeyeon Kim and Yilun Du and Sitan Chen and Bohyung Han},
  title = {{Are We Overthinking One-Step Generation?}},
  year = {2026},
  eprint = {2610.20331},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.20331},
}
Figure