A Visible and Infrared Image Fusion Framework Based on Dual-Path Encoder-Decoder and Multi-Scale Discrete Wavelet Transform

Renhe Liu, Han Wang, Shan Du, Yu LiuICIP 2023 · OpenReview

Abstract

In recent years, extensive research has been conducted on visible and infrared image fusion (VIF) task using traditional multi-scale transform-based and deep learning model-based methods. However, there is still a need to explore the combination of neural networks and multi-scale transform. This paper proposes a novel fusion framework based on a dual-path encoder-decoder and multi-scale transform. A dual-path encoder is trained to extract rich features at different depths from source images, while a shared decoder is trained to efficiently reconstruct images from the extracted feature space. We apply the discrete wavelet transform (DWT) to generate various frequency components from the extracted features. A fusion module is utilized to achieve fusion for low and high-frequency sub-bands, respectively, which is constrained by a gradient-based fusion loss function and an absolute values maximum-selection strategy. Our proposed method is superior to current state-of-the-art fusion methods, as demonstrated through quantitative and qualitative comparisons of publicly available datasets.
BibTeX
@inproceedings{liu2023visible,
  author = {Renhe Liu and Han Wang and Shan Du and Yu Liu},
  title = {{A Visible and Infrared Image Fusion Framework Based on Dual-Path Encoder-Decoder and Multi-Scale Discrete Wavelet Transform}},
  booktitle = {ICIP},
  year = {2023},
  url = {https://openreview.net/forum?id=fInyhDWIlZ},
}

Loading related content...

Figure