Non-Cross Diffusion for Semantic Consistency

Ziyang Zheng, Ruiyuan Gao, Qiang Xu

Abstract

Standard diffusion models often exhibit semantic inconsistency, where the generated content changes significantly depending on the number of sampling steps. This paper identifies "crossing flows" (XFLOW) as the primary cause—a phenomenon where distinct generative paths intersect at the same noisy state, creating ambiguous training targets. We propose Non-Cross Diffusion, which resolves these intersections by "lifting" the input dimensionality through conditioning on the model's own predicted noise. Our approach ensures that images remain semantically stable whether generated in 5 steps or 1000.

Introduction: The XFLOW Problem

Diffusion models synthesize images by reversing a flow from noise to data. Ideally, these paths should be distinct. However, in standard training, multiple generative trajectories (e.g., one for a "bird" and one for a "cat") frequently pass through the same noisy state $x_t$. At these "cross points," the model receives conflicting gradients, forcing it to target a statistical average that is often out-of-distribution.

Visualizing the XFLOW problem and the proposed Non-Cross Diffusion solution.

This ambiguity is particularly detrimental to fast sampling. When taking large steps through the latent space, the model is prone to "tripping" over these intersections and veering onto an incorrect semantic path, leading to the common issue where a low-step generation looks entirely different from a high-step one.

Methodology: Dimensionality Lifting

To untangle these intersecting flows, we apply a geometric principle: lines that cross in lower dimensions can be separated by moving into a higher-dimensional space. We achieve this by conditioning the diffusion model on its own predicted noise ($\hat{\epsilon}$).

Bootstrap Training and Architecture

Since ground-truth noise is unavailable during inference, we employ a bootstrap training strategy. The model is trained in two cases:

  1. Unconditioned (Case 1): Standard diffusion training.
  2. Conditioned (Case 2): The model receives its own previous noise estimate as an additional input.

This teaches the model to use its internal "sense of direction" to maintain a consistent trajectory. We implement this using a ControlNet-like structure, where an auxiliary branch processes the noise estimate and provides a "geometric nudge" to the main U-Net.

Training and inference pipeline for Non-Cross Diffusion using a ControlNet-like architecture.

Experimental Results

The researchers evaluated Non-Cross Diffusion across three scales: a 2D Toy Model, MNIST, and CIFAR-10. They introduced a new metric, Inference Flow Consistency (IFC), which uses PSNR to measure how well early-stage predictions align with final outputs.

Quantitative Performance

On the CIFAR-10 benchmark, the method showed significant improvements in both high-speed and high-quality regimes:

  • Fast Sampling (5 steps): Achieved an FID of 27.83, compared to 31.37 for the baseline iDDPM.
  • High Quality (1000 steps): Improved FID from 4.70 to 2.88, indicating that resolving training ambiguity helps the model learn the underlying distribution more effectively.

Qualitative Consistency

Visual results demonstrate that Non-Cross Diffusion maintains the subject's identity, pose, and color across a wide range of sampling steps (from 5 to 500), whereas baseline models often undergo semantic drift, changing the subject entirely when steps are reduced.

Conclusion and Future Work

Non-Cross Diffusion offers an elegant geometric solution to the problem of semantic drift in generative models. By resolving XFLOW through dimensionality lifting, the method achieves superior consistency and image quality. While current implementation requires retraining or fine-tuning an auxiliary branch, future research will focus on applying these constraints to large-scale pre-trained models like Stable Diffusion with minimal computational overhead.

Summary updated

BibTeX
@inproceedings{zheng2025non,
  author = {Ziyang Zheng and Ruiyuan Gao and Qiang Xu},
  title = {{Non-Cross Diffusion for Semantic Consistency}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2025},
  doi = {10.1109/wacv61041.2025.00383},
}
Figure