TL;DR
Priming Models simplify one-step generative modeling into a straightforward two-stage process: simple reconstruction pretraining (denoising at a fixed noise level) followed by distributional post-training in feature space. This "minimalist" approach achieves state-of-the-art results on ImageNet (FID 1.19) while requiring up to 15x less training compute than previous one-step methods.
What Problem Does This Paper Address?
While diffusion and flow-matching models produce stunning images, they are notoriously slow because they require multiple network evaluations (steps) to generate a single sample. This has led to a "one-step" arms race.
However, current one-step solutions are often incredibly complex. They typically rely on "distillation" (training a student model to mimic a slow teacher), complex noise trajectories, or specialized stabilization tricks. The authors of this paper ask a provocative question: Are we overthinking this? They set out to prove that you don't need complex trajectories or teacher models to train a high-quality one-step generator from scratch; you just need a better starting point.
Methods
The authors introduce a two-stage framework called Priming Models:
- Stage 1: Reconstruction Pretraining: Instead of starting from random weights, the model is first trained on a simple reconstruction task. Specifically, it learns to denoise images corrupted by a fixed amount of Gaussian noise (e.g., 80% noise). This "primes" the model to understand the basic structure and manifold of natural images.
- Stage 2: Distributional Post-training: The pretrained model is then treated as a one-step generator that maps pure Gaussian noise to images. It is refined by matching the distribution of its outputs to the distribution of real data in a frozen feature space (using an Optimal Transport/Sinkhorn loss).
Unlike prior work, this method doesn't require classifier-free guidance, complex style-conditioning, or a pretrained teacher model.
Datasets
The researchers evaluated Priming Models across three distinct and challenging domains:
- Class-conditional Image Generation: ImageNet-1K at $256 \times 256$ resolution.
- Text-to-Image Generation: CC12M and curated high-quality datasets, evaluated on GenEval and DPG-Bench.
- World Modeling (Robotics): The RT-1 and Bridge-V2 datasets, where the model predicts the next video frame based on robot actions.
Key Insights and Ideas
- The "Aha!" Moment: Distributional training (like GANs or direct matching) is notoriously difficult to start from scratch because the network initially has zero knowledge of image structure. The authors found that a simple reconstruction task is enough to entirely alleviate this optimization hurdle.
- Jacobian Alignment: Why does pretraining help so much? The authors show that pretraining aligns the model's Jacobian (how its outputs change with respect to its parameters) with the directions needed to correct the distribution. This makes the subsequent post-training updates significantly more efficient.
- Fixed Noise Suffices: Surprisingly, you don't need a complex noise schedule for pretraining. Training at a single, high noise level (like $s=0.8$) provides an effective initialization that is "close enough" to the pure noise ($s=1.0$) used during generation.
Key Results
- State-of-the-Art Quality: On ImageNet, Priming-L achieved an FID of 1.19, the best reported for any one-step generator.
- Massive Efficiency Gains: As shown in Figure 1, Priming Models reached SOTA performance using approximately 1/10th to 1/15th the training compute of prior methods like Drifting Models or WF-XL.
- Versatility: In world modeling, Priming Models outperformed the one-step "DriftWorld" baseline and even a 32-step flow-matching baseline across almost all metrics, while using significantly less compute.
- Multi-step Extension: While designed for one-step, the framework can be extended to 2 or 4 steps, further improving quality and rivaling much more expensive models.
Limitations and Future Work
- Reliance on Frozen Encoders: The distributional loss currently depends on pretrained feature encoders (like MAE or ConvNeXt). A major open question is whether this dependency can be eliminated to create a truly "from-scratch" system.
- The Performance Gap: While Priming Models are the best one-step generators, there is still a small quality gap compared to the very best multi-step diffusion models.
- Theoretical Depth: While the authors provide empirical evidence for "Jacobian alignment," a more rigorous theoretical framework for why this specific pretraining task works so well remains a direction for future research.
Summary updated