Deep Probabilistic Imaging: Uncertainty Quantification and Multi-modal Solution Characterization for Computational Imaging

He Sun, Katherine L. Bouman

Introduction

Computational imaging often involves solving ill-posed inverse problems, where multiple potential images can explain a single set of observational data. Traditional methods like Regularized Maximum Likelihood (RML) typically yield a single "best-fit" image. However, this approach fails to capture the inherent uncertainty of the reconstruction or identify alternative valid solutions (multi-modality). Deep Probabilistic Imaging (DPI) addresses this by characterizing the entire posterior distribution of possible images using an untrained, invertible generative model.

Methodology: Variational Inference with Flow-based Models

DPI treats image reconstruction as a variational inference task. Instead of learning from a large external dataset, the model optimizes the weights of a neural network for a specific set of measurements. The core of DPI is an invertible flow-based architecture (specifically Real-NVP), which allows for the analytical computation of the distribution's entropy.

The network is trained by minimizing a loss function composed of three terms:

  1. Data Fidelity: Ensures the generated images match the observed data.
  2. Prior/Regularization: Incorporates domain knowledge (e.g., image smoothness or non-negativity).
  3. Entropy: Weighted by a parameter $\beta$, this term prevents the distribution from collapsing into a single point, encouraging a diverse set of solutions.
DPI accurately reconstructs complex 2D distributions and achieves minimum KL divergence when the entropy weight is set to $\beta=1$.

Validation and Key Insights

Initial tests on toy 2D potentials—including Gaussian mixtures and sinusoidal shapes—demonstrated that DPI can capture complex, non-convex distributions. A critical finding was that setting the entropy weight $\beta=1$ results in the most accurate reconstruction of the true posterior distribution, as measured by KL divergence.

The use of flow-based models is pivotal; it makes the calculation of entropy tractable, a common bottleneck in variational inference. Furthermore, the architecture itself provides implicit regularization, favoring natural-looking images even with minimal explicit priors.

Astronomical Imaging and Uncertainty Quantification

DPI was applied to both synthetic and real data from the Event Horizon Telescope (EHT). In reconstructions of the M87* black hole, DPI not only recovered the characteristic "ring" structure but also provided a pixel-wise "confidence map." This map revealed that certain features, such as bright "knots" on the ring, carry higher uncertainty than others.

DPI reconstruction of a black hole showing the mean image alongside standard deviation and covariance maps for uncertainty quantification.

In cases with non-convex data constraints, DPI successfully identified multi-modal solutions—distinctly different images that fit the data equally well—which traditional RML methods often miss or average together. To visualize these high-dimensional posterior samples, the authors used t-SNE to cluster distinct solution modes.

Applications in Medical Imaging

Beyond astronomy, DPI was evaluated on the fastMRI dataset for compressed sensing MRI. The pixel-wise standard deviation produced by the model correlated strongly with actual reconstruction errors, providing clinicians with a reliable spatial map of where the reconstruction might be less trustworthy.

Conclusion and Future Directions

Deep Probabilistic Imaging represents a shift from point-estimate reconstruction to full posterior characterization. By providing reliable uncertainty maps and identifying multiple valid interpretations of data, DPI offers a more robust framework for scientific discovery in fields like astronomy and medical imaging. Future work aims to scale these flow-based models to higher resolutions and reduce the computational cost of per-measurement optimization.

Summary updated

BibTeX
@inproceedings{sun2021deep,
  author = {He Sun and Katherine L. Bouman},
  title = {{Deep Probabilistic Imaging: Uncertainty Quantification and Multi-modal Solution Characterization for Computational Imaging}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/aaai.v35i3.16366},
}
Figure