Autoguided Online Data Curation for Diffusion Model Training

Valeria Pais, Luis Oala, Daniele Faccio, Marco AversaarXiv 2025 · DOI · arXiv

Introduction

Training state-of-the-art diffusion models is a computationally intensive endeavor, often requiring massive datasets and significant hardware resources. As the costs of compute continue to rise, researchers are increasingly looking toward data efficiency—maximizing the performance gained from every training sample—as a viable path forward. This paper explores whether "online data curation," a technique successful in discriminative tasks, can be effectively applied to the generative modeling domain. Specifically, it introduces and evaluates Autoguided Joint Example Selection (AJEST), a framework that combines intelligent data selection with model-based guidance.

Methodology

The core of this research is the comparison of four distinct training and inference paradigms: Autoguidance, JEST, AJEST, and Random selection.

Comparison of training and inference frameworks: (a) Autoguidance, (b) JEST, (c) AJEST, and (d) Random selection.
  1. Joint Example Selection (JEST): This approach utilizes a pre-trained "reference" model to evaluate a large "super-batch" of data. It calculates a "learnability score" for each sample—defined as the difference between the loss of the current "learner" model and the loss of the reference model. The goal is to prioritize samples that are informative (hard for the learner) but not purely noisy (easy for the reference).
  2. Autoguidance: This technique focuses on the inference stage. A smaller, less-trained "guide" model works in tandem with the primary model. By analyzing the discrepancies between the two models' outputs, the system generates a corrective signal that enhances the fidelity and quality of the final generated images.
  3. AJEST: This hybrid method integrates both JEST for data selection during training and autoguidance during inference. The authors particularly emphasize Early AJEST, where data selection is prioritized during the initial phases of training when the model is most sensitive to structural learning.

Experimental Framework

The effectiveness of these methods was tested across two primary benchmarks designed to measure both structural learning and high-dimensional image generation.

Evaluation tasks: (a) A 2D synthetic tree task for measuring distribution coverage and (b) a 3x64^2 image generation task using Tiny ImageNet.
  • 2D Synthetic "Tree" Task: This task requires the model to learn a complex, branching probability distribution. It serves as a controlled environment to measure "coverage"—how effectively the model captures all parts of the target distribution without missing branches.
  • Tiny ImageNet ($3 \times 64^2$): A more realistic benchmark involving 100,000 images across 200 classes. This task evaluates the models' ability to generate coherent, high-quality images in a standard computer vision setting.

Results and Discussion

The study yields several critical insights into the trade-offs of online data curation:

  • The Overhead Challenge: While JEST improves "sample efficiency" (performance per image seen), it introduces significant computational overhead due to the extra forward passes required for scoring. In many "fixed-time" scenarios, the simple Random baseline outperformed more complex selection methods because it could process significantly more samples in the same amount of time.
  • Autoguidance as a Baseline: Autoguidance emerged as the most consistent driver of quality improvements. It provides a robust boost to image fidelity and diversity that is difficult to surpass through data selection alone.
  • Temporal Utility of Selection: Data selection is most effective in the "early game." As the learner model matures and begins to outperform the reference model, the learnability signal provided by JEST diminishes in value.
  • Performance Metrics: On Tiny ImageNet, Early AJEST showed modest improvements in perceptual metrics like FID and FD-DINOv2, particularly in very low-data regimes. However, these gains often vanished when evaluated on a strictly equal-time basis.

Conclusion

The findings suggest that while intelligent data selection like AJEST can improve sample efficiency, its practical utility is currently limited by computational overhead. For most real-world applications, the authors recommend prioritizing autoguidance to enhance model quality. Online data curation should be reserved for scenarios where training data is exceptionally scarce or where the cost of a training step vastly outweighs the cost of data selection. Future research may explore high-resolution regimes where these cost ratios might shift in favor of more aggressive selection strategies.

Summary updated

Abstract

The costs of generative model compute rekindled promises and hopes for efficient data curation. In this work, we investigate whether recently developed autoguidance and online data selection methods can improve the time and sample efficiency of training generative diffusion models. We integrate joint example selection (JEST) and autoguidance into a unified code base for fast ablation and benchmarking. We evaluate combinations of data curation on a controlled 2-D synthetic data generation task as well as (3x64x64)-D image generation. Our comparisons are made at equal wall-clock time and equal number of samples, explicitly accounting for the overhead of selection. Across experiments, autoguidance consistently improves sample quality and diversity. Early AJEST (applying selection only at the beginning of training) can match or modestly exceed autoguidance alone in data efficiency on both tasks. However, its time overhead and added complexity make autoguidance or uniform random data selection preferable in most situations. These findings suggest that while targeted online selection can yield efficiency gains in early training, robust sample quality improvements are primarily driven by autoguidance. We discuss limitations and scope, and outline when data selection may be beneficial.
BibTeX
@misc{pais2025autoguided,
  author = {Valeria Pais and Luis Oala and Daniele Faccio and Marco Aversa},
  title = {{Autoguided Online Data Curation for Diffusion Model Training}},
  year = {2025},
  eprint = {2509.15267},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2509.15267},
}
Figure