ESS-Flow: Training-free guidance of flow-based models as inference in source space

Adhithyan Kalaivanan, Zheng Zhao, Jens Sjölund, Fredrik Lindsten

Introduction

Generative models like diffusion and continuous normalizing flows have revolutionized data creation. However, applying these models to scientific and engineering domains requires "guidance"—steering the model to produce samples satisfying specific physical or chemical constraints. Traditional guidance methods often rely on gradients, which introduces significant hurdles: non-differentiability of simulators, high computational costs of backpropagating through ODE solvers, and the tendency to get trapped in local optima.

Methodology: Inference in Source Space

ESS-Flow addresses these challenges by shifting the inference problem from the complex data space to the model's latent "source space"—typically a simple standard Gaussian distribution.

Conceptual overview of ESS-Flow: Mapping target distributions from data space back to the Gaussian source space allows for efficient, Jacobian-free inference.

The core innovation lies in two areas:

  1. Jacobian-Free Formulation: The authors demonstrate that when mapping a target distribution back to the source space, the computationally expensive Jacobian terms of the flow-based model cancel out. This allows for likelihood evaluation without the heavy mathematical overhead usually associated with normalizing flows.
  2. Elliptical Slice Sampling (ESS): By operating in a Gaussian source space, the method can leverage ESS, a gradient-free MCMC algorithm. ESS explores the distribution by searching along elliptical paths, making it robust to non-differentiable constraints and capable of jumping between disconnected modes in the data manifold.

To further optimize performance, the authors implement a multi-fidelity sampling strategy. This involves using a fast, coarse ODE solver during the MCMC chain and applying a final correction with a high-fidelity solver via importance weighting.

Experimental Evaluation

The effectiveness of ESS-Flow was tested across two complex scientific domains:

  • Material Design: Guiding a FlowMM model trained on the MP-20 dataset to design crystals with specific mechanical and electronic properties (e.g., bulk modulus, band gap).
  • Protein Engineering: Using the Chroma diffusion model to reconstruct protein backbones from sparse, noisy experimental distance data.

Results and Discussion

ESS-Flow consistently outperformed gradient-based alternatives in both accuracy and physical realism. In material generation, ESS-Flow achieved a Mean Absolute Error (MAE) of 9.05 GPa for Bulk Modulus, significantly lower than the 193.86 GPa error produced by the gradient-based D-Flow method.

Comparison of D-Flow (left) and ESS-Flow (right). Gradient-based D-Flow samples often collapse into local clusters, whereas ESS-Flow successfully explores the full target distribution.

A key advantage highlighted in the results is the method's robustness to mode collapse. As illustrated above, gradient-based methods (like D-Flow) often fail to explore disconnected components of the target distribution. In contrast, ESS-Flow's gradient-free nature allows it to maintain the diversity of the prior while accurately fitting the target constraints. In protein structure prediction, this translated to structures that were not only accurate but also physically plausible, avoiding the "collapsed polymer" artifacts common in over-optimized gradient methods.

Conclusion

ESS-Flow provides a robust, training-free framework for guiding flow-based models. By performing inference in the source space, it bypasses the limitations of gradient-based guidance, offering a scalable solution for complex scientific tasks. While challenges remain regarding exact manifold constraints and the number of required forward passes, ESS-Flow represents a significant step toward practical, constraint-aware generative modeling in science.

Summary updated

Abstract

Guiding pretrained flow-based generative models for conditional generation or to produce samples with desired target properties enables solving diverse tasks without retraining on paired data. We present ESS-Flow, a gradient-free method that leverages the typically Gaussian prior of the source distribution in flow-based models to perform Bayesian inference directly in the source space using Elliptical Slice Sampling. ESS-Flow only requires forward passes through the generative model and observation process, no gradient or Jacobian computations, and is applicable even when gradients are unreliable or unavailable, such as with simulation-based observations or quantization in the generation or observation process. We demonstrate its effectiveness on designing materials with desired target properties and predicting protein structures from sparse inter-residue distance measurements.
BibTeX
@misc{kalaivanan2025ess,
  author = {Adhithyan Kalaivanan and Zheng Zhao and Jens Sjölund and Fredrik Lindsten},
  title = {{ESS-Flow: Training-free guidance of flow-based models as inference in source space}},
  year = {2025},
  eprint = {2510.05849},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2510.05849},
}
Figure