Abstract
Image fusion integrates complementary information from multi-source images to generate more informative results. Recently, the diffusion model, which demonstrates unprecedented generative potential, has been explored in image fusion. However, these approaches typically incorporate predefined multimodal guidance into diffusion, failing to capture the dynamically changing significance of each modality, while lacking theoretical guarantees. To address this issue, we reveal a significant spatio-temporal imbalance in image denoising; specifically, the diffusion model produces dynamic information gains in different image regions with denoising steps. Based on this observation, we Dig into the Diffusion Information Gains (Dig2DIG) and theoretically derive a diffusion-based dynamic image fusion framework that provably reduces the upper bound of the generalization error. Accordingly, we introduce diffusion information gains (DIG) to quantify the information contribution of each modality at different denoising steps, thereby providing dynamic guidance during the fusion process. Extensive experiments on multiple fusion scenarios confirm that our method outperforms existing diffusion-based approaches in terms of both fusion quality and inference efficiency.
Introduction
In computer vision, image fusion aims to combine complementary features from multiple sources—such as structural heat signatures from infrared sensors and rich textures from visible-light cameras—into a single, high-quality composite. While Diffusion Models (DMs) have emerged as powerful tools for this task, most current methods utilize "fixed guidance," assuming the relative importance of source images remains constant throughout the denoising process.
However, research reveals a critical spatio-temporal imbalance in denoising. Early stages of diffusion (high noise) prioritize structural cues, often dominant in infrared images, while later stages (low noise) require fine-grained textures typically found in visible images. Fixed-weight approaches often fail to balance these needs, leading to blurred textures or lost structural targets. Dig2DIG addresses this by dynamically "digging" into the information gains of each modality at every step.
Methodology
The core of the Dig2DIG framework is the Diffusion Information Gains (DIG) metric, which provides a theoretical and practical basis for dynamic weighting.
1. Quantifying Information Gain
For each source modality $c_k$, the DIG metric quantifies its contribution at timestep $t$ by measuring the discrepancy reduction between a one-step denoised version and the source itself. High DIG indicates that a modality is particularly informative at that specific stage of reconstruction.
2. Dynamic Weighting and Inference
DIG values are transformed via a softmax function into time-dependent weights $w_k(t)$, which scale the guidance gradients during the reverse diffusion process. This ensures the model prioritizes structural reconstruction early and texture refinement late. Notably, Dig2DIG is a zero-shot, training-free framework that can be applied to pre-trained diffusion models during inference.
3. Theoretical Guarantee
The authors provide a formal proof demonstrating that the generalization error of the fusion operator is bounded. By maximizing the covariance between dynamic weights and the ideal guidance direction (approximated by DIG), the framework mathematically ensures a reduction in the upper bound of generalization error.
Experimental Results
Dig2DIG was evaluated across three primary fusion tasks: Visible-Infrared Fusion (VIF), Multi-Focus Fusion (MFF), and Multi-Exposure Fusion (MEF), using benchmarks such as M3FD and MSRS.
- Performance: Dig2DIG consistently outperformed state-of-the-art methods (e.g., DDFM, SwinFusion) across metrics including SSIM, MSE, and LPIPS. It effectively avoids "ghosting" artifacts and maintains high fidelity even in extreme lighting conditions.
- Efficiency: By updating weights at intervals (hyperparameter $S$), the method achieves significant speedups. On the M3FD dataset, Dig2DIG completed inference in 52 seconds, compared to 633 seconds for the CCF model, without sacrificing quality.
- Downstream Utility: Fused images were tested with a YOLOv5 detector, showing marked improvements in pedestrian detection accuracy, confirming the practical value of the enhanced information integration.
Conclusion
Dig2DIG introduces a mathematically grounded, dynamic approach to diffusion-based image fusion. By recognizing and leveraging the spatio-temporal imbalance of denoising through the DIG metric, the framework achieves superior visual quality and computational efficiency. Future work will focus on addressing weight variance in high-noise regimes and extending the method to handle misaligned or severely degraded source images in real-world scenarios.
Summary updated