Evolvable Conditional Diffusion

Zhao Wei, Chin Chun Ooi, Abhishek Gupta, Jian Cheng Wong, Pao-Hsiung Chiu, Sheares Xue Wen Toh, Yew-Soon Ong

TL;DR

This paper introduces "Evolvable Conditional Diffusion," a novel method that allows diffusion models to be guided by non-differentiable, black-box physics simulators. This breakthrough enables autonomous scientific discovery by leveraging existing, complex simulation tools without requiring them to be differentiable, a major hurdle in current AI-driven design.

What Problem Does This Paper Address?

Current state-of-the-art guided diffusion models excel at generating samples that meet specific criteria. However, a significant limitation arises when these criteria need to be evaluated by complex, real-world physics simulators (like those used in fluid dynamics or electromagnetics). These simulators are often "black-boxes" – they provide an output (e.g., pressure drop, transmission profile) but their internal workings are not differentiable. This lack of differentiability prevents the direct use of their evaluation results to guide the diffusion model's generation process, which typically relies on gradients. This forces researchers to either train expensive, differentiable surrogate models or abandon the use of high-fidelity simulators altogether, hindering the application of diffusion models in many scientific domains.

Methods

The core innovation is an "evolvable conditional diffusion" approach that reframes the guidance process through the lens of probabilistic evolution. Instead of relying on gradients from a differentiable fitness function, the method treats the diffusion model's denoising distribution as a population that evolves.

Here's a breakdown:

  • Black-Box Optimization: The guidance is formulated as an optimization problem where the goal is to maximize a desired fitness function.
  • Evolutionary Principles: The method draws inspiration from evolutionary computation. The denoising distribution is updated based on the "fitness" of samples drawn from it.
  • Gradient-Free Update: Crucially, the update rule for the denoising distribution's mean is derived from first principles. It turns out to be mathematically analogous to the gradient-based update used in conventional guided diffusion models, but it's estimated using samples and their evaluated fitness values, completely bypassing the need for derivatives of the fitness function.
  • Fitness Shaping: To improve robustness and handle extreme fitness values, a rank-based fitness shaping mechanism is employed, replacing raw fitness scores with their ranks.

The algorithm iteratively samples from the current denoising distribution, evaluates the fitness of these samples using the black-box solver, updates the distribution's parameters based on these fitness values, and then samples again for the next denoising step.

Datasets

The paper does not explicitly mention training datasets for the diffusion models themselves. Instead, it focuses on the evaluation of generated designs using existing, non-differentiable solvers. For the experiments, the authors used:

  • Fluidic Channel Topology Design: Evaluated using steady-state incompressible Navier-Stokes (N-S) equations (simulated by a black-box solver, with a regressor trained to mimic its output for proof-of-concept).
  • Meta-Surface Design: Evaluated using computational electromagnetics solvers (like Ansys HFSS, with a regressor trained to mimic its output for proof-of-concept).

Key Insights and Ideas

  • Bridging the Gap: The most significant insight is the successful integration of non-differentiable, high-fidelity physics simulators into the guided diffusion framework. This opens up a vast landscape of scientific problems previously inaccessible to such methods.
  • Evolvability as a Principle: Framing the guidance process through "evolvability" provides a principled way to derive a gradient-free update rule that is both effective and mathematically sound.
  • Analogy to Gradients: The surprising finding that the derived gradient-free update rule is mathematically equivalent to gradient-based updates under certain assumptions is a key theoretical contribution. It shows that the evolutionary approach naturally converges to a similar solution space as gradient-based methods, but without the strict differentiability requirement.
  • Robustness to Noise: The evolutionary approach, by sampling from a population and using rank-based fitness shaping, is inherently more robust to noisy evaluations compared to methods relying on precise gradient estimates.

Key Results

The proposed "Evolvable Conditional Diffusion" method was validated in two AI for Science scenarios:

  • Fluidic Channel Topology Design: The method successfully generated fluidic channel designs with significantly reduced pressure drop compared to unconditional diffusion. The guidance effectively steered the generation towards designs that better satisfied the optimization objective.
  • Meta-Surface Design: Similarly, the method generated meta-surface designs that better matched target transmission profiles, resulting in lower Mean Absolute Error (MAE) compared to unconditional diffusion.

In both cases, increasing the guidance scaling factor (α) further biased the generation towards designs that better met the specific criteria, demonstrating the method's effectiveness and controllability. The results showed that even applying guidance for a shorter duration (e.g., the last 10 steps) yielded significant improvements.

Limitations and Future Work

  • Computational Cost: While the method avoids differentiable proxies, it still requires evaluating multiple samples per denoising step using the black-box solver. This can be computationally intensive, especially for complex simulations.
  • Scalability: The paper notes that scalability depends on the "intrinsic" dimensionality of the problem, which is often lower than the full input dimensionality. However, extremely high-dimensional problems might still pose challenges.
  • Future Work: The authors suggest that this approach can be extended to other domains requiring optimization through non-differentiable evaluations. The paper implicitly points towards further exploration of optimizing the sampling and evaluation strategies to improve computational efficiency. The integration with autonomous laboratories is also highlighted as a future direction.

Summary updated

Abstract

This paper presents an evolvable conditional diffusion method such that black-box, non-differentiable multi-physics models, as are common in domains like computational fluid dynamics and electromagnetics, can be effectively used for guiding the generative process to facilitate autonomous scientific discovery. We formulate the guidance as an optimization problem where one optimizes for a desired fitness function through updates to the descriptive statistic for the denoising distribution, and derive an evolution-guided approach from first principles through the lens of probabilistic evolution. Interestingly, the final derived update algorithm is analogous to the update as per common gradient-based guided diffusion models, but without ever having to compute any derivatives. We validate our proposed evolvable diffusion algorithm in two AI for Science scenarios: the automated design of fluidic topology and meta-surface. Results demonstrate that this method effectively generates designs that better satisfy specific optimization objectives without reliance on differentiable proxies, providing an effective means of guidance-based diffusion that can capitalize on the wealth of black-box, non-differentiable multi-physics numerical models common across Science.
BibTeX
@inproceedings{wei2025evolvable,
  author = {Zhao Wei and Chin Chun Ooi and Abhishek Gupta and Jian Cheng Wong and Pao-Hsiung Chiu and Sheares Xue Wen Toh and Yew-Soon Ong},
  title = {{Evolvable Conditional Diffusion}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2025},
  url = {https://openreview.net/forum?id=O5CRXr4X8b},
}
Figure