Dig2DIG: Dig into Diffusion Information Gains for Image Fusion

Baoshuo Cai, Bing Cao, Changqing Zhang, Qinghua HuSubmission to ICLR 2026 · Rejected

TL;DR

Current diffusion-based image fusion methods treat source modalities uniformly across time, ignoring that different visual features emerge at distinct stages of denoising. Dig2DIG introduces Diffusion Information Gains (DIG) to dynamically measure how much unrecovered information remains in each modality, provably tightening the model's generalization error bound. Without requiring any fine-tuning or training, Dig2DIG achieves state-of-the-art fusion quality across diverse tasks while cutting diffusion sampling runtime by roughly 70%.

What Problem Does This Paper Address?

Image fusion merges complementary inputs—such as visible and infrared images, or multi-exposure/multi-focus captures—into a single composite image with rich details and salient targets. While diffusion models generate realistic images, existing diffusion fusion methods usually apply static or heuristic weights across all denoising steps.

However, diffusion models exhibit a pronounced spatio-temporal imbalance: coarse structures and thermal hotspots converge during early denoising stages, whereas high-frequency textures (like visible backgrounds) appear much later. Forcing equal or fixed guidance at every step can dilute informative features, bias the fused output toward weak modalities, and waste computational resources.

Methods

Dig2DIG formulates dynamic fusion directly within the reverse sampling process of a standard, pre-trained unconditional diffusion model without additional training:

  • Generalization Error Upper Bound: The authors theoretically prove that the generalization error of guided diffusion fusion decomposes into a constant term minus the covariance between modality guidance weights ($w_k$) and the residual, not-yet-fused modality information ($I_{k,t}$). Maximizing this positive covariance provably minimizes the upper bound on generalization error.
  • Diffusion Information Gain (DIG): Because true residual information is latent and unobservable at test time, Dig2DIG uses the single-step reconstruction error of each modality as a computable proxy: $$\text{DIG}_k(t) = \ell(\hat{c}_k^t, c_k)$$ where $\hat{c}_k^t$ is the one-step denoised estimate of noisy input $c_k^t$, and $\ell$ is an image distance metric (such as $\ell_2$).

  • Dynamic Guidance Weighting: The weights for multimodal score guidance are derived per region and timestep using a softmax over the DIG values ($w_k(t) \propto \exp(\text{DIG}_k(t))$). Modalities with higher remaining information gain receive stronger guidance.

  • Efficient Sampling Schedule: Instead of calculating DIG at every step, DIG is refreshed every $S = 10$ steps, and sampling is executed in just 25 steps (using larger steps at high noise levels and smaller steps near clean data).
   Source Modalities (e.g., IR & Visible)
                    │
   [ Add Noise & Single-Step Denoise ]
                    │
       Compute Residual Proxy (DIG)
                    │
         Softmax Normalization (w_k)
                    │
                    ▼
     Dynamic Multimodal Score Guidance
                    │
       Reverse Diffusion Update (x_{t-1})

Datasets

The authors evaluated the method across three distinct fusion paradigms using existing public benchmarks:

  • Visible-Infrared Image Fusion (VIF): LLVIP, M3FD, and MSRS.
  • Multi-Focus Image Fusion (MFF): MFFW, plus MFI-WHU (used for ground-truth validation of the residual proxy).
  • Multi-Exposure Image Fusion (MEF): MEFB, and a multi-exposure dataset with $K > 2$ exposures (Cai et al., 2018).

No new datasets were introduced.

Key Insights and Ideas

  • Theoretical Grounding for Dynamic Fusion: Rather than relying on heuristic rules, the authors demonstrate mathematically that the effectiveness of dynamic weighting depends on positive covariance between fusion weights and residual modality information.
  • DIG as an Empirical Mirror of Residual Info: On the ground-truth MFI-WHU benchmark, DIG exhibited a Pearson correlation of $r = 0.9345$ with true ground-truth residual error, validating DIG as a reliable, observable proxy for unobserved residual information.
  • Both Spatial and Temporal Adaptivity Matter: Ablation showed that applying DIG across both space (regions) and time (denoising steps) significantly outperformed applying it across time alone, because different image regions within the same timestep require guidance from different modalities.

Key Results

  • Visible-Infrared Fusion: On the LLVIP dataset, Dig2DIG attained the top scores across five metrics, achieving an MSE of 1464 (compared to the next best, CCF, at 1658) and an SSIM of 1.23. On M3FD and MSRS, it achieved the highest SSIM (1.41 and 1.42) and lowest MSEs.
  • Multi-Focus & Multi-Exposure Fusion: On MFFW, Dig2DIG achieved a Standard Deviation (SD) of 72.95 (surpassing CCF's 69.71) and leading Entropy (7.87) and Spatial Frequency (22.60). On MEFB, it achieved the top SD (75.05) and Average Gradient (8.10).
  • Inference Speedup: On M3FD, Dig2DIG completed inference in 52 seconds (819 TFLOPS), compared to 180 seconds (2820 TFLOPS) for DDFM and 633 seconds (8505 TFLOPS) for CCF, delivering a ~70% computational savings over baseline diffusion techniques.
  • Downstream Utility: Pedestrian detection evaluations with YOLOv5 on LLVIP confirmed that the enhanced contrast and sharpness translated to superior object detection in low-light environments.

Limitations and Future Work

  • Noise Sensitivity in Early Steps: At very high noise levels ($t \approx T$), single-step reconstruction error has high variance, making early DIG estimates noisy and necessitating larger initial step sizes.
  • Residual Vanishing in Late Steps: As $t \to 0$, the denoised estimate approaches the clean input, causing residual differences to become very small; updating DIG at every late step degrades performance.
  • Diffusion Latency vs. CNNs: Although Dig2DIG is significantly faster than competing diffusion methods (25 vs. 100–300 steps), iterative diffusion inference remains noticeably slower than single-pass CNN or Transformer models, posing a challenge for real-time edge deployment.

Summary updated

Abstract

Image fusion integrates complementary information from multiple sources to generate more informative results. Recently, the diffusion model, which demonstrates unprecedented generative potential, has been explored in the context of image fusion. During diffusion model generation, information emerges at unequal rates, so the fusion should dynamically weight the source modalities. To address this issue, we reveal a significant spatio-temporal imbalance in image denoising; specifically, the diffusion model produces dynamic information gains in different image regions with denoising steps. Based on this observation, we dive into the Diffusion Information Gains (DIG) and theoretically derive a diffusion-based dynamic image fusion framework that provably reduces its upper bound of the generalization error. Accordingly, we introduce diffusion information gains to quantify the information contribution of each modality at different denoising steps, thereby providing dynamic guidance during the fusion process. Experiments on multiple fusion scenarios confirm that our method outperforms existing diffusion-based approaches in terms of both fusion quality and inference efficiency.
BibTeX
@misc{cai2025dig,
  author = {Baoshuo Cai and Bing Cao and Changqing Zhang and Qinghua Hu},
  title = {{Dig2DIG: Dig into Diffusion Information Gains for Image Fusion}},
  year = {2025},
  eprint = {2503.18627},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2503.18627},
}

Loading related content...

Figure