General Intelligent Imaging and Uncertainty Quantification by Deterministic Diffusion Model

Weiru Fan, Xiaobin Tang, Yiyi Liao, Da‐Wei WangarXiv 2024 · DOI · arXiv

TL;DR

Researchers have developed a Deterministic Diffusion Model (DDM) that solves a major headache in computational imaging: reconstructing high-fidelity images from "nonlocal" patterns (like laser speckle) where traditional deep learning fails. By replacing the random noise of standard diffusion models with a deterministic path and embedding Bayesian inference, DDM achieves state-of-the-art reconstruction while providing a "reliability map" of its own predictions.

What Problem Does This Paper Address?

Computational imaging (CI) is the art of using algorithms to "see" what a standard lens cannot—such as looking through a scattering medium (like fog or biological tissue) or capturing nonlinear optical signals.

The challenge lies in nonlocal patterns. In a standard photo, a pixel on your sensor corresponds to a specific spot on the object. In many CI tasks, however, light from one point on the object is scattered across the entire sensor. Traditional Deep Learning (DL) architectures, like CNNs, rely on "local receptive fields," meaning they are great at textures but struggle to piece together these global-to-local scrambles.

While Diffusion Probabilistic Models (DPMs) have shown promise in image generation, they are notoriously slow, computationally expensive, and—most importantly—stochastic. In high-stakes fields like medical imaging or autonomous driving, a model that generates a "plausible but fake" image due to random noise is a liability. We need models that are both precise and capable of telling us when they are unsure.

Methods

The authors introduce the Deterministic Diffusion Model (DDM). Here is how it differs from the standard diffusion models (like Stable Diffusion) you might know:

  1. Deterministic Degeneration: Instead of degrading an image into pure Gaussian noise (as DPMs do), DDM uses a linear "rectified straight flow." It progressively transforms the ground truth image into the actual raw, scrambled data captured by the detector.
  2. Multi-step Denoising: Rather than trying to solve the reconstruction in one giant leap (which leads to blurred edges), DDM breaks the task into $T$ small, manageable steps. At each step, the model "infuses" the raw pattern back into the prediction, compensating for lost information.
  3. Bayesian Uncertainty Quantification: The authors replaced standard deterministic weights with a Bayesian Neural Network (BNN) using Monte Carlo dropout. By running the inference multiple times with random weight dropouts, the model produces not just an image, but a variance map. This allows the system to quantify "data uncertainty" (noise in the system) and "model uncertainty" (limitations of the training).

Datasets

The model was trained and evaluated using the COOS-7 (Cells Out Of Sample) dataset, which contains over 132,000 images of mouse cells.

  • Processing: The researchers stacked protein and nucleus channels to create structurally complex targets.
  • Physical Simulation: These images were then used to generate synthetic and experimental nonlocal patterns via two physical processes: Second Harmonic Generation (SHG) and Strong Scattering through a multimode fiber (MMF).

Key Insights and Ideas

  • The "Adiabatic" Advantage: From a physics perspective, DDM acts like an adiabatic process. By changing the state slowly over many steps, the neural network only needs to recover a tiny component of the image at each stage, preventing the massive information loss seen in single-step CNNs.
  • Data Augmentation by Design: Because the model learns to reconstruct the image from any point $t$ in the diffusion path, a dataset of $N$ images effectively becomes a dataset of $N \times T$ samples. This significantly reduces the risk of overfitting and lowers model uncertainty.
  • The Perceptual Paradox: The authors noted that while DDM might sometimes have a higher Mean Squared Error (MSE) than "smoother" models, its perceptual quality (sharpness, texture, and structural similarity) is vastly superior. This highlights the inadequacy of simple pixel-wise metrics for evaluating advanced imaging models.

Key Results

  • Superior Fidelity: DDM outperformed state-of-the-art frameworks including CNNs with nonlocal modules, Invertible Restoring Autoencoders (IRAE), and conditional DPMs (Palette/ControlNet).
  • Efficiency: DDM achieved high-quality results in just 100 steps, whereas standard DPMs often required 2000 steps to reach similar levels of coherence in SHG tasks.
  • Reliable Uncertainty Maps: The BNN successfully identified that uncertainty is highest at edges and sharp regions of the cells. This provides a crucial "trust metric" for researchers using these images for diagnosis or analysis.

Limitations and Future Work

While DDM is a leap forward, the authors acknowledge a few hurdles:

  • The Detail-Confidence Trade-off: There is an intrinsic trade-off where more diffusion steps provide richer detail but can slightly decrease the statistical confidence in those specific fine-grained features.
  • Metric Optimization: Future work could involve embedding the "scheduling" ($\alpha_t$) directly into the neural network as a learnable parameter to find the mathematically optimal path for different types of physical scattering.
  • Expansion: The framework is currently optimized for 2D images, but the authors suggest it could be extended to voice, video, and 3D point clouds by modifying the degeneration function.

Summary updated

Abstract

Computational imaging is crucial in many disciplines from autonomous driving to life sciences. However, traditional model-driven and iterative methods consume large computational power and lack scalability for imaging. Deep learning (DL) is effective in processing local-to-local patterns, but it struggles with handling universal global-to-local (nonlocal) patterns under current frameworks. To bridge this gap, we propose a novel DL framework that employs a progressive denoising strategy, named the deterministic diffusion model (DDM), to facilitate general computational imaging at a low cost. We experimentally demonstrate the efficient and faithful image reconstruction capabilities of DDM from nonlocal patterns, such as speckles from multimode fiber and intensity patterns of second harmonic generation, surpassing the capability of previous state-of-the-art DL algorithms. By embedding Bayesian inference into DDM, we establish a theoretical framework and provide experimental proof of its uncertainty quantification. This advancement ensures the predictive reliability of DDM, avoiding misjudgment in high-stakes scenarios. This versatile and integrable DDM framework can readily extend and improve the efficacy of existing DL-based imaging applications.
BibTeX
@misc{fan2024general,
  author = {Weiru Fan and Xiaobin Tang and Yiyi Liao and Da‐Wei Wang},
  title = {{General Intelligent Imaging and Uncertainty Quantification by Deterministic Diffusion Model}},
  year = {2024},
  eprint = {2408.13061},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2408.13061},
}
Figure