Abstract
Solving inverse problems—recovering a clean signal from corrupted measurements—is a fundamental challenge in computer vision. While diffusion models have emerged as powerful priors for this task, existing posterior sampling methods often suffer from approximation errors that lead to "drift" and reduced reconstruction fidelity. This paper introduces a novel information-theoretic refinement by maximizing the Conditional Mutual Information (CMI) between the measurement and the reconstructed signal throughout the reverse diffusion process. By leveraging Hutchinson’s trace estimation to handle high-order derivatives, the proposed method serves as a universal, plug-and-play enhancement for existing diffusion solvers, significantly improving image sharpness and structural accuracy across standard benchmarks like FFHQ and ImageNet.
1. Introduction
Inverse problems, such as deblurring, super-resolution, and inpainting, involve recovering an original image $x_0$ from a noisy, degraded measurement $y$, typically modeled as $y = Ax_0 + n$. Diffusion Models (DMs) have revolutionized this field by providing a robust generative prior $p(x_0)$. To solve an inverse problem, one ideally samples from the posterior distribution $p(x_0|y)$. However, the exact gradient of the log-likelihood $\nabla_{x_t} \log p(y|x_t)$ required for this sampling is mathematically intractable during the intermediate steps of the diffusion process.
Current state-of-the-art methods, such as Diffusion Posterior Sampling (DPS), rely on approximations to guide the denoising process. While effective, these approximations can cause the sampling trajectory to deviate from the measurement constraints, resulting in "hallucinated" artifacts or blurry reconstructions. This paper addresses this limitation by introducing a guidance term based on Conditional Mutual Information (CMI).
2. Methodology: CMI Maximization
The core contribution of this work is the integration of an information-theoretic objective into the reverse diffusion loop. The authors propose maximizing the CMI, $I(x_0; y | x_t)$, at each denoising step. This objective ensures that the latent state $x_t$ retains the maximum possible information about the original measurement $y$ as it evolves toward the clean reconstruction $x_0$.
2.1 The Gaussian Approximation
To derive a tractable gradient for CMI, the authors assume that the conditional distribution $p(x_0|x_t)$ can be approximated as a multivariate normal distribution. This simplification is a common convention in diffusion literature and allows for the derivation of a closed-form expression for the CMI gradient.
2.2 Efficient Computation via Hutchinson’s Trace Estimation
A significant technical challenge arises because the gradient of the CMI involves a third-order derivative of the log-density, specifically $\nabla^3_{x_t} \log p_t(x_t)$. For high-dimensional data like $256 \times 256$ images, computing this directly is computationally prohibitive, scaling at $O(d^3)$.
To overcome this, the researchers employ Hutchinson’s trace estimation. This stochastic technique approximates the trace of a matrix using random vectors and Hessian-vector products. By utilizing modern automatic differentiation frameworks, the authors can compute these high-order terms efficiently on standard GPUs, making the CMI guidance feasible for practical applications.
3. Experimental Setup and Results
The proposed CMI-based guidance was evaluated as a "plugin" for several existing solvers, including DPS and $\Pi$GDM. Experiments were conducted on the FFHQ (Face Forensics HQ) and ImageNet datasets at $256 \times 256$ resolution.
3.1 Tasks and Metrics
The method was tested against five corruption types:
- Random Inpainting: 90% of pixels removed.
- Box Inpainting: Central region masked.
- Gaussian and Motion Blur: Simulating camera and movement artifacts.
- Super-Resolution: $4\times$ upscaling.
Performance was measured using Fréchet Inception Distance (FID) for image quality, Learned Perceptual Image Patch Similarity (LPIPS) for perceptual accuracy, and Structural Similarity Index (SSIM).
3.2 Key Findings
The results demonstrate that CMI enhancement consistently improves performance across nearly all tasks and baselines:
- Quantitative Superiority: CMI-enhanced versions of DPS and $\Pi$GDM almost universally outperformed their vanilla counterparts in FID, LPIPS, and SSIM metrics.
- Qualitative Fidelity: Visual inspections revealed that the CMI term effectively corrects the "drift" seen in standard DPS. In tasks like $16\times$ super-resolution, the reconstructions exhibited significantly sharper details and fewer artifacts.
- Robustness in Ill-Posed Scenarios: The method proved particularly resilient in extreme cases, such as 90% random masking, where the measurement $y$ provides very little information.
4. Discussion and Limitations
While the CMI approach offers a powerful tool for improving inverse problem solvers, it introduces certain trade-offs. The use of Hutchinson’s trick requires multiple random samples per denoising step to ensure accurate trace estimation, which increases the total inference time compared to vanilla solvers. Additionally, the current mathematical derivation is optimized for linear inverse problems ($y = Ax + n$). Extending this framework to non-linear forward models remains a subject for future research. Finally, the CMI guidance introduces a step-size hyperparameter $\eta_t$ that requires careful tuning to balance the prior and the measurement information.
5. Conclusion
This paper presents a compelling information-theoretic framework for diffusion-based inverse problem solving. By treating information retention as a geometric constraint on the sampling path, the authors provide a universal enhancement that improves both the mathematical rigor and the visual quality of image reconstructions. The successful application of Hutchinson’s trace estimation to high-order derivatives opens new avenues for incorporating complex information-theoretic objectives into generative modeling.
Summary updated