A Novel Synchronized Fusion Model for Multi-Band Images

Bin Wang, Guifeng Bai, Suzhen Lin, Yanbo Wang, Jianchao ZengIEEE Access 2019 · OpenReview

Abstract

The purpose of this paper is to address the problem of synchronous multi-band image fusion, which encompasses far-infrared images (FIRI), near-infrared images (NIRI), and visible images (VISI). By studying the imaging principles for each of these kinds of image, we find that there are similarities between NIRI and the final fusion image, including both thermal radiation and the reflection of light. NIRI is therefore used as the reference image and relevant FIRI or VISI features are used to enhance the thermal target, background and edge texture, thereby making fusion possible. The paper begins by establishing an optimization model for multi-band image synchronous fusion, based on imaging characteristics. This is followed by a more detailed elaboration of the model. An alternating direction method of multipliers (ADMM) is then used to construct an algorithm to solve the model according to the specific circumstances of its use. After this, an experimental design is analyzed to acquire the main parameters of the model. Finally, a number of experiments are used to demonstrate that the fusion produces good results in terms of both visual effects and objective evaluation indicators.

Introduction

In fields like remote sensing, security, and military reconnaissance, the ability to synthesize a single, high-quality image from multiple sensors is critical. Traditional methods often rely on dual-band fusion, typically combining one infrared image with one visible image. However, modern imaging systems frequently capture data across three or more bands: Far-Infrared (FIRI) for thermal heat, Near-Infrared (NIRI) for reflected light capable of penetrating haze, and Visible (VISI) light.

The current standard approach fuses these bands sequentially (e.g., fusing FIRI and VISI, then fusing the result with NIRI). This "daisy-chain" method is computationally expensive and prone to information loss or artifact introduction at each stage. This paper proposes a "synchronized" optimization model that treats all bands as part of a single, unified optimization problem, avoiding the pitfalls of sequential processing.

Methodology

The core innovation lies in the use of NIRI as a physical "bridge." Because NIRI images capture both thermal radiation characteristics (similar to FIRI) and reflected light characteristics (similar to VISI), they serve as an ideal reference anchor for the fusion process.

The Synchronized Fusion Model

The proposed model minimizes an energy function $E(u)$ consisting of three primary terms:

  1. Fidelity Term: Ensures the fused image $u$ remains consistent with the NIRI reference ($I_N$).
  2. Intensity Enhancement Term: Extracts prominent features, such as thermal targets from FIRI and background textures from VISI, by selecting pixels with higher intensity.
  3. Gradient Term: Fuses edges and textures from all bands to maintain sharpness.
Overview of the synchronized fusion model and the iterative ADMM solution process.

The model utilizes the $l_1$ norm rather than the standard $l_2$ norm. The $l_1$ norm is particularly effective for handling sparse data like image gradients and is robust against the non-Gaussian noise typical of infrared sensors. To solve this optimization problem efficiently, the authors employ the Alternating Direction Method of Multipliers (ADMM) combined with Fast Fourier Transforms (FFT), enabling rapid convergence as shown in the iterative steps of the fusion process.

Experimental Results

The model was evaluated using the TNO Image Fusion Dataset, focusing on diverse outdoor scenes like "Nato Camp" and "Soldier in Trench." Performance was benchmarked against state-of-the-art methods including MSVD, DTCWT-SR, and GTF.

Quantitative Performance

Quantitative analysis showed that the proposed model consistently outperformed competitors across several metrics:

  • Entropy (Ent) and Mutual Information (MI): Higher scores indicated superior information retention from source images.
  • Average Gradient (Grad) and Spatial Frequency (SF): Improvements in these metrics confirmed that the fused images were objectively sharper.

Qualitative Clarity

Visually, the synchronized model produced high-contrast images where thermal targets (e.g., human figures) remained distinct while background details (e.g., trees and buildings) were preserved with high clarity. Unlike sequential methods, it avoided unnatural darkening of the sky or blurring of critical edges.

Conclusion

By shifting from sequential to synchronized multi-band fusion, this research provides a more robust framework for multi-sensor imaging. The use of NIRI as a reference bridge and the application of the $l_1$ norm via ADMM results in images that are both information-rich and visually clear. Future work will explore adaptive parameter selection to automate the tuning of the model for varying environmental conditions and expand the application to medical and hyperspectral imaging domains.

Summary updated

Abstract

The purpose of this paper is to address the problem of synchronous multi-band image fusion, which encompasses far-infrared images (FIRI), near-infrared images (NIRI), and visible images (VISI). By studying the imaging principles for each of these kinds of image, we find that there are similarities between NIRI and the final fusion image, including both thermal radiation and the reflection of light. NIRI is therefore used as the reference image and relevant FIRI or VISI features are used to enhance the thermal target, background and edge texture, thereby making fusion possible. The paper begins by establishing an optimization model for multi-band image synchronous fusion, based on imaging characteristics. This is followed by a more detailed elaboration of the model. An alternating direction method of multipliers (ADMM) is then used to construct an algorithm to solve the model according to the specific circumstances of its use. After this, an experimental design is analyzed to acquire the main parameters of the model. Finally, a number of experiments are used to demonstrate that the fusion produces good results in terms of both visual effects and objective evaluation indicators.
BibTeX
@inproceedings{wang2019novel,
  author = {Bin Wang and Guifeng Bai and Suzhen Lin and Yanbo Wang and Jianchao Zeng},
  title = {{A Novel Synchronized Fusion Model for Multi-Band Images}},
  booktitle = {IEEE Access},
  year = {2019},
  url = {https://openreview.net/forum?id=R6Jrvj9THt},
}
Figure