Abstract
This paper proposes an unsupervised multi-exposure image fusion (MEF) method via contrastive learning, termed as MEF-CL. It breaks exposure limits and performance bottlenecks faced by existing methods. MEF-CL designs similarity constraints to preserve contents in source images, eliminating the need for artificial ground truth and avoiding the negative impacts of inappropriate training targets. By exploring a latent feature space and applying contrastive learning, the model guides fused images to approximate normal-light samples while staying away from inappropriately exposed ones. Consequently, MEF-CL is applicable to image pairs of any multiple exposures rather than just complementary pairs. Qualitative and quantitative experiments validate its superiority in illumination, texture detail, and color saturation.
Introduction: The Ground Truth Trap and Exposure Limits
Digital cameras often fail to capture the full dynamic range of a scene in a single shot. Multi-exposure image fusion (MEF) addresses this by merging multiple images taken at different exposure levels. However, existing deep learning methods face two major hurdles:
- The Ground Truth Trap: Supervised methods require "ground truth" images that do not exist in nature. These are usually created artificially (pseudo-GT), leading to models that inherit the biases and artifacts of the creation process.
- Exposure Limits: Most unsupervised methods rely on pixel-level similarity to source images, effectively acting as "interpolators." This forces the fused result to stay between the exposures of the source images, typically requiring a perfectly complementary pair (one under-exposed and one over-exposed). If both source images are under-exposed, the result remains dark.
Methodology: The MEF-CL Framework
The authors propose MEF-CL, an end-to-end unsupervised network consisting of two main components: a Fusion Network and a Contrastive Learning Block.
Fusion Network
The core is a 10-layer fully connected DenseNet architecture. It takes two source images as input and produces a fused image. DenseNet was selected for its high feature utilization and ability to prevent information loss during deep processing.
Contrastive Learning Block
This is the framework's "secret sauce." Using a pre-trained VGG-16 network, the model extracts features from the fused image (the anchor), "positive" samples (naturally captured normal-light images), and "negative" samples (the improperly exposed source images). In the latent feature space, the loss function ($L_{InfoNCE}$) "pulls" the fused image closer to the normal-light distribution and "pushes" it away from the under/over-exposed source distributions.
To ensure the result remains true to the scene, the authors combine this contrastive loss with Mean Square Error (MSE) to maintain intensity distribution and Structural Similarity (SSIM) to preserve textures and structural details from the source images.
Breaking the Exposure Limit
Unlike traditional MEF models that find a middle ground, MEF-CL acts as an "extrapolator." Because it understands the characteristics of a high-quality image via contrastive learning, it can take two dark images and produce a well-lit result by moving toward the "normal-light" cluster in feature space.
As shown in the comparative results, MEF-CL successfully enhances illumination even when both source images are under-exposed, a scenario where existing methods like U2Fusion typically produce dark, unusable results. Similarly, it can correct over-exposed pairs by pulling them toward the normal-light distribution.
Experimental Results
MEF-CL was evaluated using the SICE dataset and compared against nine state-of-the-art (SOTA) methods, including traditional algorithms (GFF) and deep learning models (U2Fusion, MEF-GAN, TransMEF).
- Quantitative Performance: MEF-CL achieved top scores in MEF-SSIM (structural similarity), CC (correlation coefficient), and DeltaE (color accuracy).
- Qualitative Quality: Visual comparisons show that MEF-CL produces images with more saturated colors and fewer artifacts (like halos or ghosting) compared to supervised methods held back by pseudo-GT training data.
- Efficiency: The model is highly efficient, processing an image pair in approximately 0.033 seconds, making it suitable for real-time applications.
Conclusion
By alleviating the dependence on source image exposure levels and artificial ground truth, MEF-CL shows superior generalization across various scenes. The use of contrastive learning in a latent feature space allows the model to transcend the "interpolation" bottleneck of previous unsupervised methods, delivering images with appropriate illumination, detailed textures, and vivid colors regardless of the input exposure levels.
Summary updated