The Entropic Signature of Class Speciation in Diffusion Models

Florian Handke, Dejan Stancevic, Felix Koulischer, Thomas Demeester, Luca Ambrogioni

Generating an image with a diffusion model is often compared to a sculptor revealing a statue from a block of marble. At the start, there is only pure noise; by the end, a clear subject has emerged. However, this transition is not a uniform, gradual process. Recent theoretical work from statistical physics suggests the existence of a "speciation" window—a narrow regime where the model breaks symmetry and commits to a specific semantic class (e.g., deciding the noise will become a "tiger shark" rather than an "eft").

While high-level theories describe this phenomenon, researchers have lacked a reliable, operational way to measure this "commitment" in real-world, trained models. Without tracking semantic emergence, it remains difficult to fully understand why techniques like Classifier-Free Guidance (CFG) work or how to optimize sampling schedules for better control. To bridge this gap, recent research introduces class-conditional entropy as a practical tool to pinpoint exactly when a diffusion model makes its semantic decisions.

Tracking the Entropic Signature

To monitor the denoising process, the authors propose measuring the class-conditional entropy—the uncertainty about the final class label given the current noisy state.

  1. Theoretical Foundation: Using Gaussian Mixture Models (GMMs), the researchers demonstrate mathematically that the temporal derivative of this entropy (the "entropy production") peaks exactly during the speciation transition.
  2. Partitioned Entropy: To analyze specific features rather than just broad classes (like the 1,000 ImageNet categories), the authors introduce "partitioned" entropy. By creating binary partitions (e.g., "Is this a wooden chair, or a wooden chair with a cat?"), they can resolve semantic decisions at varying levels of abstraction.
  3. Iterative Posterior Estimation: Because the "true" posterior of a trained model is inaccessible, the method relies on an iterative update mechanism. By comparing the predictions of conditional and unconditional denoisers at each step, the framework propagates and estimates class probabilities backward along the denoising trajectory.

Key Insights: How Models "Think"

The application of class-conditional entropy reveals several fundamental behaviors of diffusion models across synthetic data, ImageNet-512 (using EDM2-XS), and LAION-5B (using Stable Diffusion 1.5 and SDXL 1.0).

  • The "Speciation" Window is Real: Semantic structure does not emerge gradually. Instead, there is a specific "aha!" moment for the model, represented by a sharp peak in entropy production.
  • Coarse-to-Fine Emergence: Different features resolve at different times. Low-frequency features, such as global color or broad spatial layouts (e.g., "blue walls"), resolve much earlier in the denoising process (at higher noise levels) than high-frequency, localized details (e.g., the presence of a "cat" or a "window"). For instance, in EDM2-XS, the coarse structure of a "tiger shark" resolves at higher noise levels than the fine structure of a "paintbrush."
  • Guidance as a Time-Shifter: One of the most significant findings is the mechanism behind Classifier-Free Guidance (CFG). CFG does not merely "strengthen" the signal; it redistributes information over time. Guidance effectively pushes the speciation window to earlier, noisier stages, forcing the model to commit to a class sooner. The authors demonstrated that the "optimal" guidance intervals align perfectly with peaks in entropy production. Applying guidance after the entropy has already collapsed provides little to no benefit for semantic structure.
  • Architecture Dynamics (VP vs. VE): The paper highlights a fundamental difference between model architectures. In Variance-Preserving (VP) models like Stable Diffusion, the speciation transition is remarkably sharp. In Variance-Exploding (VE) models like EDM, the transition window broadens as the dimensionality of the data increases. Furthermore, when comparing text-to-image models, the entropic signatures were significantly cleaner and smoother on SDXL than on SD 1.5, suggesting that better-trained models exhibit more stable semantic emergence.

Limitations and Future Directions

The primary limitation of this approach is its reliance on an online posterior update mechanism. This estimation assumes the model is well-calibrated; if the model makes systematic errors across noise levels, the resulting entropy profiles can become distorted.

Currently, this framework serves as a powerful "diagnostic" tool to observe when speciation occurs, rather than a direct "control" law. However, it lays the groundwork for future innovations. The authors suggest that these entropic signals could eventually be used to dynamically adjust guidance scales or sampling steps in real-time, paving the way for faster, more controllable generation. Integrating external "observer" models to detect specific features during the entropy estimation pipeline could further refine this time-localized control.

Summary updated

Abstract

Diffusion models do not recover semantic structure uniformly over time. Instead, samples transition from semantic ambiguity to class commitment within a narrow regime. Recent theoretical work attributes this transition to dynamical instabilities along class-separating directions, but practical methods to detect and exploit these windows in trained models are still limited. We show that tracking the class-conditional entropy of a latent semantic variable given the noisy state provides a reliable signature of these transition regimes. By restricting the entropy to semantic partitions, the entropy can furthermore resolve semantic decisions at different levels of abstraction. We validate our method on EDM2-XS and Stable Diffusion 1.5, where class-conditional entropy consistently isolates the noise regimes critical for semantic structure formation. Finally, we use our framework to quantify how guidance redistributes semantic information over time. Together, these results connect information-theoretic and statistical physics perspectives on diffusion and provide a principled basis for time-localized control.
BibTeX
@inproceedings{handke2026entropic,
  author = {Florian Handke and Dejan Stancevic and Felix Koulischer and Thomas Demeester and Luca Ambrogioni},
  title = {{The Entropic Signature of Class Speciation in Diffusion Models}},
  booktitle = {International Conference on Machine Learning},
  year = {2026},
  url = {https://openreview.net/forum?id=rp02mO31RL},
}

Loading related content...

Figure