Resolving Ambiguity in Inverse Problems with Deep Priors

20 concepts21 papers1 proposal

Imagine looking at a shadow cast on a wall and trying to reconstruct the exact 3D object that created it. This is an inverse problem: you observe an effect (the shadow) and must infer the underlying cause (the object).

In the physical and data sciences, inverse problems are everywhere. We try to reconstruct a high-resolution image from a few compressed measurements, map the subsurface layers of the Earth from surface seismic waves, or determine the internal electrical properties of a beating heart from sensors on the chest.

Mathematically, these problems are often ill-posed or underdetermined. Because the forward process compresses or degrades information, it possesses a non-trivial null space—a set of features that are completely wiped out during observation. As a result, an infinite number of distinct, mathematically valid candidate solutions can produce the exact same observed data. This is the challenge of solution ambiguity.

To resolve this ambiguity, we must introduce a prior—additional information that tells us which solutions are realistic and which are merely mathematical artifacts. Over the last decade, a profound shift has occurred: researchers have moved away from rigid, hand-crafted mathematical priors toward flexible, data-driven deep generative priors.


The Geometry of Ambiguity

To understand why deep priors work, it helps to visualize the geometry of an underdetermined linear inverse problem:

$$y = Ax + \eta$$

where $y$ is our sparse observation, $A$ is the forward measurement operator, $x$ is the high-dimensional true signal, and $\eta$ is noise. Because $A$ maps a high-dimensional space to a lower-dimensional one, the equation $y = Ax$ defines a high-dimensional linear subspace (a hyperplane) of candidate solutions. Every single point on this hyperplane perfectly satisfies the observed data.

Historically, the standard way to pick a single point from this hyperplane was to assume sparsity. If we assume the true signal has very few non-zero coefficients in some basis (like a wavelet transform), we can use algorithms like Lasso to find the sparsest candidate. Other approaches used structured mathematical priors, such as the submodular mixture models proposed by El Halabi et al., to capture clusters or tree-like structures in the data.

However, real-world signals—such as natural images, physical fluid fields, or anatomical structures—are far more complex than simple sparsity or hand-crafted rules can capture. They lie on intricate, non-linear manifolds that traditional mathematics struggles to define explicitly.


Constraining Solutions to Learned Manifolds

The core insight of modern deep inverse frameworks is simple: let a neural network learn the manifold of valid solutions.

In a seminal paper, Bora et al. proposed replacing traditional sparsity assumptions entirely with pre-trained generative models (such as VAEs or GANs). Instead of searching the entire high-dimensional space for a vector $x$ that satisfies $y = Ax$, they restricted the search to the range of a generator $G(z)$, where $z$ is a low-dimensional latent vector:

$$\min_{z} \| A G(z) - y \|_2^2$$

By optimizing in the low-dimensional latent space $z$ rather than the high-dimensional pixel space, the optimization is geometrically constrained to the manifold of realistic images. The infinite sea of mathematically valid but physically absurd solutions lying outside this manifold is completely ignored. The authors demonstrated that this approach could recover signals with 5 to 10 times fewer measurements than standard sparse recovery techniques.

Traditional Inverse Problem:
Search Space: Entire High-Dimensional Vector Space
[ Infinite Hyperplane of Valid Solutions ] ---> Can yield noisy, unrealistic artifacts

Generative Prior Inverse Problem:
Search Space: Low-Dimensional Latent Space z ---> Generator G(z) ---> Learned Manifold
[ Intersection of Hyperplane & Manifold ] ---> Yields a unique, realistic solution

What if you do not have access to a clean, external dataset of ground-truth signals to pre-train this generator? Recent advancements show that networks can learn these manifolds directly from compressed or noisy data. For instance, Qin introduced a ground-truth-free meta-learning framework (MetaCS) that bridges the gap between unsupervised and supervised compressive sampling. Similarly, Böck et al. combined the mathematical rigor of Sparse Bayesian Learning with deep generative architectures (CSVAE and CSGMM) to reconstruct high-quality signals without ever seeing uncompressed ground-truth data during training.


From Single Estimates to Posterior Sampling

While finding a single "best" reconstruction is useful, it can be fundamentally misleading when a problem is highly ambiguous. If three completely different configurations can explain the data equally well, picking just one hides the inherent risk of the decision. In fields like remote sensing (Heestermans Svendsen et al.) or engineering design (Keshavarzzadeh et al.), quantifying this uncertainty is vital.

This realization has driven a shift from point estimation to posterior sampling. Instead of finding a single point where the hyperplane intersects the manifold, the goal is to map out the entire probability distribution of plausible solutions—the posterior distribution $p(x|y)$.

To achieve this stochastically, Kawar et al. introduced SNIPS (Solving Noisy Inverse Problems Stochastically). By coupling Langevin dynamics with a pre-trained Gaussian denoiser, their algorithm draws diverse, sharp, and physically consistent samples from the posterior. Each sample represents a different, valid interpretation of the degraded input, making the inherent uncertainty of the recovery process explicitly visible.

Taking a different probabilistic route, Pang et al. leveraged self-supervised Bayesian deep learning by treating the neural network's weights themselves as random variables. This allows the network to naturally capture noise and solution ambiguity for image recovery without requiring any external training data.


Physics-Driven Operators as Implicit Regularizers

When we move from pixel spaces to physical systems governed by partial differential equations (PDEs), the nature of the ambiguity changes. The solutions must not only look realistic; they must obey the laws of physics.

In these scenarios, deep networks are designed to act as implicit regularizers by learning the underlying physical operators. For example, Vadeboncoeur et al. developed the Physics-Driven Deep Latent Variable Model (PDDLVM). By treating the PDE residual as a probabilistic observation of zero, their framework simultaneously solves forward and inverse problems while providing coherent uncertainty estimates. To handle situations where the physical domain or geometry changes, Vadeboncoeur et al. later introduced geometry-aware autoencoder priors (GABI), enabling full physical field reconstructions from sparse sensor data across varying shapes.

When the exact underlying physical laws are unknown or too complex to write down, attention mechanisms can be used to discover them while managing ambiguity. Yu et al. proposed the Nonlocal Attention Operator (NAO), which learns data-dependent kernel maps across multiple physical systems. This architecture acts as an automatic regularizer, allowing zero-shot generalization to unseen physical states. Similarly, Liu et al. combined Fourier Neural Operators with attention mechanisms to discover hidden physical laws from sparse, noisy data.

These physics-informed deep priors are unlocking solutions to highly non-linear, high-dimensional inverse problems that were previously computationally intractable:

  • Geophysics Inversion: Reconstructing complex subsurface velocity or conductivity maps from surface seismic waves (Feng et al.).
  • Cardiac Personalization: Inferring hidden internal tissue properties and electrical pathways of a patient's heart from sparse surface-level measurements (Dhamala et al.).
  • Subsurface Permeability: Estimating rock permeability from fluid flow rates using conditional Sinkhorn GANs to map out high-dimensional latent spaces (Chen et al.).

By restricting the optimization landscape to learned manifolds and physically consistent spaces, deep generative priors provide a mathematically rigorous yet flexible framework to navigate, quantify, and ultimately resolve the fundamental ambiguity of inverse problems.

Key Foundational Work

Map estimation for Bayesian mixture models with submodular priors (2014) — contributed 1 concept elements to this direction.
Compressed Sensing using Generative Models (2017) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_ill_posed_inverse_problems_solution_ambi_0967ea30,
  title = {ill-posed inverse problems (solution ambiguity)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/ill-posed-inverse-problems-solution-ambiguity/rdr_0967ea3078724336924171039eb77c1f},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_ill_posed_inverse_problems_solution_ambi_0967ea30_v1,
  title = {ill-posed inverse problems (solution ambiguity)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/ill-posed-inverse-problems-solution-ambiguity/rdr_0967ea3078724336924171039eb77c1f/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake