An inverse problem asks us to reconstruct an underlying, unobserved signal $x$ from a set of noisy measurements $y = A(x) + \epsilon$. Because the forward operator $A$ often discards crucial information—such as during downsampling, blurring, or incomplete tomographic projections—these problems are fundamentally ill-posed. Infinitely many candidate signals could explain the same observed data.
Historically, we resolved this ambiguity by injecting hand-crafted mathematical assumptions into the reconstruction process, such as enforcing smoothness or sparsity. However, as reviewed by Ongie et al. in their taxonomy of deep learning techniques for imaging (Deep Learning Techniques for Inverse Problems in Imaging), a powerful paradigm shift has occurred. Instead of relying on rigid, hand-designed formulas, modern frameworks use deep generative models to learn the complex distribution of valid data directly from examples, serving as highly expressive, data-driven priors.
The Geometry of the Prior: Manifold and Latent Space Constraints
To understand why deep generative priors work so well, it helps to look at them geometrically. Consider a generative model like a Variational Autoencoder (VAE), a Generative Adversarial Network (GAN), or a normalizing flow. These networks are trained to map a simple, low-dimensional latent space to a complex, high-dimensional data distribution. In doing so, they effectively trace out a low-dimensional manifold of plausible solutions embedded within the massive space of all possible signals.
When solving an inverse problem, we can restrict our search exclusively to this learned manifold. If a solution does not lie on the manifold, the model considers it physically impossible and rejects it.
High-Dimensional Signal Space
┌─────────────────────────────────────────┐
│ │
│ • Noisy/Ill-posed Candidate │
│ \ │
│ \ (Projection onto Manifold) │
│ ▼ │
│ ╭──────────────────────────╮ │
│ ╱ • Plausible Solution ╲ │
│ │ │ │
│ ╲ Learned Data Manifold ╱ │
│ ╰──────────────────────────╯ │
└─────────────────────────────────────────┘
This structural constraint acts as a powerful regularizer. For example, Borsoi et al. demonstrate this in hyperspectral unmixing (Deep Generative Endmember Modeling: An Application to Unsupervised Spectral Unmixing), showing that restricting the latent space of a generative model to a very low dimension forces it to capture primary physical factors of variability (like moisture or lighting) while remaining robust against noise and overfitting.
Similarly, Serianni et al. utilize a low-rank latent space constraint within a deep autoencoder to solve the severely ill-posed problem of diffuse optical tomography (Learnable Priors Support Reconstruction in Diffuse Optical Tomography). By constraining the latent space, the model inherently promotes smoother, more robust interpolations and suppresses measurement noise.
This manifold constraint is highly versatile. In shape reconstruction and obstacle scattering, Vlasic et al. integrate a normalizing flow to learn a manifold of plausible shapes (Implicit Neural Representation for Mesh-Free Inverse Obstacle Scattering). By confining the optimization to this learned shape manifold, the inversion process becomes significantly more stable, efficient, and resilient to limited-view artifacts than traditional physics-only approaches.
From Optimization to Bayesian Inference
While finding a single best-fit solution on a manifold is valuable, physical applications often demand uncertainty quantification. We do not just want a single reconstructed image; we want to know how much we can trust its details. This requires moving from deterministic regularization to a fully Bayesian framework.
As explained by Oliviero-Durmus et al. (Generative modelling meets Bayesian inference: a new paradigm for inverse problems), fusing deep generative models with Bayesian inference allows us to replace hand-crafted priors with data-driven distributions that capture intricate, real-world structures. This fusion yields highly accurate posterior distributions, reflecting both the physics of the forward model and the statistical reality of the data.
To practically sample from these high-dimensional posterior distributions, researchers leverage advanced probabilistic techniques:
- Denoising Score-Matching: Ramzi et al. employ denoising score-matching to train a neural network to estimate the score—the gradient of the log-probability density—of the data distribution (Denoising Score-Matching for Uncertainty Quantification in Inverse Problems). By embedding this learned score prior into an annealed Hamiltonian Monte Carlo (HMC) sampling routine, the authors successfully reconstruct complex MRI data and provide pixel-wise uncertainty maps.
- Proximal Nested Sampling: For complex model selection tasks, Liaudat et al. extend Proximal Nested Sampling (PNS) to accommodate data-driven priors learned by deep networks (Proximal Nested Sampling with Data-Driven Priors for Physical Scientists). This allows physical scientists to perform rigorous Bayesian model selection and high-dimensional imaging without being bottlenecked by the simplicity of traditional priors.
Ultimately, deep generative priors succeed because they decouple the representation of the data from the physics of the measurement. By letting deep networks learn what a valid solution looks like, and letting classical physics or Bayesian sampling handle how that solution maps to observations, this approach provides a robust, interpretable, and highly accurate framework for solving the world's hardest inverse problems.
Key Foundational Work
Denoising Score-Matching for Uncertainty Quantification in Inverse Problems (2020) — contributed 3 concept elements to this direction.
Deep Learning Techniques for Inverse Problems in Imaging (2020) — contributed 2 concept elements to this direction.
Deep Generative Endmember Modeling: An Application to Unsupervised Spectral Unmixing (2019) — contributed 1 concept elements to this direction.