Abstract: Given an inverse problem with a normalizing flow prior, we wish to estimate the distribution of the underlying signal conditioned on the observations. We approach this problem as a task of conditional inference on the pre-trained unconditional flow model. We first establish that this is computationally hard for a large class of flow models. Motivated by this, we propose a framework for approximate inference that estimates the target conditional as a composition of two flow models. This formulation leads to a stable variational inference training procedure that avoids adversarial training. Our method is evaluated on a variety of inverse problems and is shown to produce high-quality samples with uncertainty quantification. We further demonstrate that our approach can be amortized for zero-shot inference.
Introduction
In many scientific and imaging fields, we encounter inverse problems: the task of recovering an original signal $x$ from a partial or noisy observation $y$. Mathematically, this is expressed as sampling from the conditional distribution $p(x|y)$. While deep generative models like GANs and VAEs have served as effective priors for these tasks, Normalizing Flows (NFs) are particularly compelling due to their invertibility and ability to provide exact likelihood evaluations.
However, this research identifies a significant theoretical hurdle: for a large class of flow models—specifically those utilizing additive coupling layers like RealNVP or Glow—exact conditional sampling is NP-hard. By reducing the problem to a 3-SAT instance, the authors demonstrate that as models scale in complexity, finding the exact reconstruction becomes computationally intractable. This finding shifts the research focus from exact to efficient approximate inference.
Methodology: Composing Flows
To address the hardness of exact inference, the authors propose a framework called Composing Normalizing Flows. The core idea is to avoid direct inversion of the complex, pre-trained "base" flow $f$. Instead, they introduce a secondary, smaller flow known as a pre-generator $\hat{f}$.
1. Compositional Framework
The final signal is modeled as a composition: $x = f(\hat{f}(\epsilon))$, where $\epsilon$ is simple Gaussian noise. The pre-generator maps this noise into the latent space of the base model, which then maps it to the image space. This structure allows the model to search the manifold of the pre-trained prior more effectively.
2. Variational Inference and Smoothing
The pre-generator is trained using a Variational Inference (VI) objective. Unlike GAN-based approaches, this avoids the instabilities of adversarial training. A critical technical contribution is the introduction of a smoothing parameter $\sigma$. In inverse problems, requiring an exact match to the observation $y$ can lead to divergent loss functions. By treating the observation as a "soft" Gaussian constraint, the authors derive a loss function that balances a KL-divergence term (maintaining the naturalness of the distribution) with a Mean Squared Error (MSE) term (ensuring data fidelity).
3. Amortization
For repetitive tasks like image inpainting, the pre-generator can be "amortized." By training it to accept the observation $y$ as an input, the model enables "zero-shot" inference. Once trained, the model can reconstruct new, unseen images instantly without requiring per-image optimization.
Results and Uncertainty Quantification
The framework was evaluated on MNIST, CIFAR-10, and CelebA-HQ (64x64) datasets. A key advantage of this method over point-estimate techniques is its ability to recover the entire conditional distribution rather than just a single "best guess."
As shown in the figure above, the model provides explicit uncertainty quantification. In regions where the image data is missing (such as the chin or mouth area), the model generates multiple plausible reconstructions. The associated histograms visualize the entropy and multi-modality of the posterior distribution at specific pixel locations, showing exactly where the model is confident versus where multiple valid interpretations exist.
Quantitatively, the proposed method outperformed baselines such as Langevin Monte Carlo (LMC) and Ambient VI in terms of realism (FID) and perceptual similarity (LPIPS). Furthermore, the authors demonstrated that the Minimum Mean Square Error (MMSE) estimator—derived by averaging multiple samples from the composed flow—consistently achieves higher PSNR than standard point-estimate methods.
Conclusion
By composing a pre-trained flow with a variational pre-generator, this work provides a stable and efficient path for solving inverse problems. The method not only circumvents the computational hardness of exact conditional inference but also provides a rigorous framework for uncertainty quantification. While the sensitivity to the smoothing hyperparameter $\sigma$ remains a challenge for future automation, the speed and quality of the amortized inference represent a significant step forward for practical generative priors.
Summary updated