When we train a generative diffusion model, we celebrate its ability to produce diverse outputs from a single prompt. This creative diversity stems from a fundamental design choice: the model maps a simple, high-dimensional Gaussian noise distribution to the complex manifold of real-world data. By sampling a different random noise vector (or "seed") at the start of the reverse process, the model traverses a different trajectory and arrives at a completely unique image.
However, when diffusion models are deployed for downstream engineering, perception, or high-stakes diagnostic tasks—such as image super-resolution, medical imaging, or semantic segmentation—this creative diversity becomes a critical failure mode. If the same low-resolution input or brain scan can yield wildly different reconstructions depending purely on an arbitrary random seed, the model lacks the reliability required for practical adoption.
To solve this, a major research direction has emerged focused on diagnosing and eliminating this stochastic variance, transforming probabilistic generative models into stable, deterministic, and highly reproducible systems.
The Root Cause: Initial Noise Sensitivity and Trajectory Divergence
To understand why generative models exhibit such high variance, we must look at the geometry of the generation trajectory. Whether using Stochastic Differential Equations (SDEs) or Ordinary Differential Equations (ODEs), the reverse denoising process can be viewed as navigating a complex, high-dimensional vector field. The initial noise vector acts as the boundary condition for this system.
Because this vector field contains numerous local minima and sharp decision boundaries ("basins of attraction"), even microscopic variations in the initial noise seed can cause the trajectory to diverge drastically. As noted by Luoqian Jiang et al. in Restabilizing Diffusion Models with Predictive Noise Fusion Strategy for Image Super-Resolution, this stochastic instability means that identical conditioning inputs can yield reconstructions ranging from high-fidelity images to those riddled with severe hallucinations or artifacts.
This sensitivity is so pronounced that it introduces uncontrollable variance in video generation (Huayi Xu et al. in Optimal Noise pursuit for Augmenting Text-to-Video Generation) and even undermines the reliability and reproducibility of adversarial attacks (Jingzheng Li et al. in OBJVanish: Physically Realizable Text-to-3D Adv. Generation of LiDAR-Invisible Objects).
Stochastic SDE Path: [Initial Noise Seed A] ──(Random Fluctuations)──> [Output Variant 1]
[Initial Noise Seed B] ──(Random Fluctuations)──> [Output Variant 2]
Deterministic Flow: [Prior-Aware Input] ────────────────────────────> [Consistent Output]
To stabilize these models, researchers have developed three core strategies: optimizing the initial boundary conditions, transitioning to deterministic flow matching, and utilizing trajectory fusion.
Strategy 1: Optimizing the Boundary Conditions
If drawing a random seed from a standard Gaussian distribution is a gamble, why not find or calculate the "perfect" starting point? One prominent branch of solutions focuses on replacing pure randomness with optimized or prior-aware initializations.
Instead of treating the initial noise as an arbitrary starting point, the authors of Solving Diffusion ODEs with Optimal Boundary Conditions for Better Image Super-Resolution pre-calculate an "approximately optimal boundary condition" for solving the diffusion ODE. By mathematically tying the starting noise to the degraded input data, they bypass the random fluctuations that plague standard sampling.
Other approaches treat the noise space as a searchable landscape. For instance, Association for Artificial Intelligence 2026 et al. propose an evolutionary framework in Dual-Seed Evolutionary Algorithm for Noise Optimization in Diffusion Models to search the noise landscape for "golden seeds" that maximize semantic alignment.
For practical image editing tasks like bokeh rendering, Shiyue Yan et al. demonstrate in Reschedule Diffusion-based Bokeh Rendering that a fixed, prior-aware starting point derived directly from the input image drastically reduces variance, ensuring consistent and reproducible aesthetic effects across multiple executions.
Strategy 2: Transitioning to Deterministic Flow Matching
While optimizing the initial seed helps, traditional diffusion models often inject noise at every intermediate step of the reverse SDE trajectory, compounding the instability. To completely eliminate this source of randomness, researchers are increasingly replacing SDEs with deterministic ODEs via Flow Matching and Rectified Flows.
Flow Matching models the generation process as a straight, deterministic trajectory from the noise distribution to the data distribution. By avoiding the random wandering inherent to SDEs, the path becomes entirely predictable. For example, in Steering the Flow: Inverting Face Recognition Models via Gradient-Guided Flow Matching, the authors leverage Flow Matching to maintain precise control over identity reconstruction, preventing the random deviations caused by stochastic noise injection.
This deterministic formulation allows generative models to be safely integrated into strict perception pipelines:
- Semantic Segmentation: In SemFlow: Binding Semantic Segmentation and Image Synthesis via Rectified Flow, Chaoyang Wang et al. use reversible ODEs to bridge semantic segmentation and image synthesis, removing the pixel-level uncertainty and artifacts that occur when stochastic models try to output discrete labels.
- Time-Series Imputation: Hangtian Wang et al. apply this to time-series imputation in Time-Gated Multi-Scale Flow Matching for Time-Series Imputation, ensuring that critical medical or financial records are reconstructed identically every time.
Strategy 3: Trajectory Fusion and Hybrid Sampling
A third approach recognizes that while stochasticity can be useful for exploration or detail generation, it must be tamed or averaged out to achieve stability.
One intuitive way to eliminate individual sample artifacts is through ensembling. In Perception-Distortion Trade-off in the SR Space Spanned by Flow Models, A. Murat Tekalp et al. show that fusing or averaging multiple random samples from a generative flow model cancels out stochastic noise, allowing users to navigate the trade-off between high perceptual realism and strict fidelity to the ground truth.
Alternatively, instead of averaging full samples at the end, one can restrict or guide the stochasticity during the generation process itself. Junzhe Li et al. introduce this hybrid concept in MixGRPO: Unlocking Flow-based GRPO Efficiency with Mixed ODE-SDE. By confining stochastic optimization to a narrow "sliding window" of denoising steps and utilizing deterministic solvers everywhere else, the authors prevent the accumulation of grainy artifacts and maintain a highly stable probability path.
Alternative Paradigms: Moving Beyond Diffusion
When absolute determinism is required, some researchers argue that the diffusion framework itself should be abandoned in favor of architectures with built-in stability.
In Moving Beyond Diffusion: Hierarchy-to-Hierarchy Autoregression for fMRI-to-Image Reconstruction, Xu Zhang demonstrates that a hierarchical autoregressive approach completely bypasses the need for random noise seeds. By mapping brain signals directly to multi-scale image tokens, this method delivers reconstructions that are inherently deterministic and significantly faster, proving vital for clinical diagnostics where stability is paramount.
Key Foundational Work
Improving Diffusion Models for Inverse Problems using Manifold Constraints (2022) — contributed 1 concept elements to this direction.
Text-guided mask-free local image retouching (2022) — contributed 1 concept elements to this direction.