Standard diffusion models operate by iteratively peeling away layers of noise. In the classic formulation, the neural network is trained to predict the precise noise vector ($\epsilon$) added to an image at a given timestep. While mathematically elegant for learning score functions, this noise-prediction paradigm treats the intermediate states as abstract, noisy blends that are difficult to interpret or manipulate.
A powerful alternative paradigm has emerged: re-parameterizing or training the diffusion model to directly predict the final, clean target state ($x_0$) at every denoising step. Shifting the objective from noise estimation to direct clean state prediction unlocks significant advantages in guidance flexibility, structural fidelity, and generation speed.
Standard Diffusion: Noisy State (x_t) ---> Predict Noise (ε) ---> Next Noisy State (x_{t-1})
Clean Prediction: Noisy State (x_t) ---> Predict Clean (x_0) ---> Next Noisy State (x_{t-1})
|
v
[Apply Clean-Data Guidance / Constraints]
The Look-Ahead Preview: Guidance Without Retraining
One of the most compelling reasons to predict the clean state $x_0$ at each step is that it provides a "look-ahead preview" of the final image. This preview is incredibly useful when you want to guide the diffusion process using external, pre-trained predictive models.
Most external models—such as safety classifiers, aesthetic predictors, or biological neural simulators—are trained exclusively on clean, natural images. If you attempt to guide a standard diffusion model by feeding an intermediate noisy state $x_t$ into these classifiers, they fail because the noise corrupts the features they rely on. To fix this, you would normally have to retrain the guidance models on noisy data distributions.
Direct $x_0$ prediction completely bypasses this requirement. By estimating the terminal clean image state at each step, the system can pass this clean preview directly to standard, frozen predictive models.
- Biological Fidelity: Pierzchlewicz et al. leverage this in Energy Guided Diffusion (EGG) to synthesize images that excite specific brain cells. By calculating gradients relative to the estimated clean state, they utilize predictive models trained only on clean data, accelerating generation by nearly 5x without retraining the guidance network.
- Data Mining and Safety: Similarly, Hayden et al. use clean-state estimation to guide diffusion for mining longtail edge cases in production machine learning, ensuring accurate guidance signals by presenting the predictive model with data matching its original training distribution. For content moderation, Na et al. introduce Safe Text embedding Guidance (STG), evaluating a safety function on the expected final image at each step to filter out harmful content or unauthorized styles before the generation is complete.
- Semantic Synchronization: When stitching together multiple local diffusion processes to create large panoramas, operating on noisy states can cause structural misalignments because noise masks the underlying layout. Lee et al. demonstrate that synchronizing processes via a perceptual similarity loss on the predicted clean images ensures global stylistic and structural consistency, eliminating the "Frankenstein effect" of disjointed boundaries.
Preserving Structure and Geometry in Constrained Tasks
For image restoration and conditional generation—such as super-resolution, inpainting, or medical segmentation—the target distribution is highly constrained by the input condition. In these domains, predicting the clean image directly is often far more effective than predicting noise.
Noise is inherently unconstrained and chaotic; predicting it can cause the model to lose track of fine structural details. When a model directly predicts $x_0$, the optimization objective simplifies. It allows the network to be optimized using standard reconstruction losses (like Mean Absolute Error) directly tied to the target data distribution.
- Image Super-Resolution: In single-image super-resolution, Niu et al. show that a clean-image objective simplifies optimization and yields fewer artifacts. In a follow-up work on accelerated super-resolution, Niu et al. emphasize that because the target distribution is tightly bounded by the low-resolution input, an $x_0$-prediction target constrains the model to valid image information much better than noise residuals can, drastically reducing the required inference steps.
- Text and Cultural Artifact Preservation: Fine details, like the precise strokes of text characters or intricate cultural patterns, are easily obliterated during iterative noise removal. Zhu et al. apply direct target image prediction to blind text image inpainting, finding that it significantly enhances text legibility and character recognition accuracy by preventing structural distortion. Similarly, Zou et al. utilize an $x_0$-prediction objective to preserve strict compositional rules and authentic color palettes when generating Chinese intangible cultural heritage artwork.
- Enforcing Physical and Geometric Filters: When generating data with strict real-world constraints, predicting $x_0$ at each step allows you to apply structural filters directly during the generation loop. For instance, Lu et al. design a human motion prediction framework that predicts the clean state at each step to facilitate the immediate application of anatomical topology and skeletal joint constraints. In medical imaging, Monzon et al. configure a spine segmentation model to directly predict clean segmentation masks rather than noise patterns, optimizing the output with composite overlap and boundary alignment losses.
Accelerating Generation to a Single Step
The traditional multi-step denoising process is a major computational bottleneck. However, if a student model is trained to compress this entire trajectory, predicting noise becomes counter-intuitive because there are no subsequent steps left to subtract that noise.
To achieve one-step generation, researchers re-parameterize models to map the initial noise vector directly to the final clean data space.
Multi-Step (Noise Objective): x_T ---> x_{T-1} ---> ... ---> x_1 ---> x_0
One-Step (Clean Objective): x_T ------------------------------------> x_0
- Diffusion GAN Hybrids: Xu et al. introduce UFOGen, which hybridizes diffusion models with a GAN-based objective. By re-parameterizing the generator to directly predict the final clean image while sampling intermediate states from the forward process, they stabilize one-step text-to-image synthesis.
- Knowledge Distillation: In the context of consistency models and distillation, Nguyen et al. develop SwiftBrush, a one-step text-to-image generator. They re-parameterize the student network to directly predict the final clean image, effectively bridging the gap between the iterative teacher model and the single-step requirement of the student.
Internal Learning and Inherent Interpretability
Beyond guidance and speed, direct $x_0$ prediction offers unique advantages in data-scarce environments and model interpretability.
When training data is limited to a single image—a paradigm known as internal learning—the model must quickly capture local asset correlations without drifting. Almansa et al. demonstrate that a lightweight model configured to directly predict the clean image state simplifies the objective function enough to make internal learning for image inpainting highly effective and computationally cheap. Similarly, the authors of RAW-Diffusion find that directly predicting a denoised RAW image from an RGB guide simplifies the generative mapping, allowing the model to produce high-fidelity sensor data without massive dataset requirements.
Finally, direct clean-state prediction makes the inner workings of the model far more transparent. Bernold et al. integrate inherently interpretable B-cos networks into diffusion models. They discover that training the model to predict the original image $x_0$ instead of the noise leads to significantly more faithful explanations. It allows humans to clearly see which parts of a text prompt are influencing specific regions of the emerging image at any point in the process, offering a superior balance for interpretability.
Key Foundational Work
ACDMSR: Accelerated Conditional Diffusion Models for Single Image Super-Resolution (2023) — contributed 2 concept elements to this direction.
Text Image Inpainting via Global Structure-Guided Diffusion Models (2024) — contributed 2 concept elements to this direction.
Diffusion-based image inpainting with internal learning (2024) — contributed 1 concept elements to this direction.