TL;DR
This paper introduces a novel two-exposure image fusion method that overcomes limitations of existing techniques, particularly when dealing with images that are both underexposed or overexposed. By generating an enhanced intermediate virtual image using Optimized Adaptive Gamma Correction (OAGC), the method significantly improves the contrast, saturation, and well-exposedness of the final fused image, leading to better detail preservation and overall visibility.
What Problem Does This Paper Address?
Capturing high-dynamic-range (HDR) scenes often requires combining multiple images taken at different exposure values (EVs). While traditional multi-exposure fusion (MEF) methods work well with sequences of images having small EV differences, they struggle with scenarios involving only two exposures, especially when both images are underexposed or overexposed. This limitation arises because existing methods often rely on interpolating between input images or assume at least one image has well-exposed regions. This paper addresses the challenge of fusing two images with potentially large EV differences, where neither input might contain sufficient well-exposed content, leading to poor fusion results with lost details.
Methods
The core of this work lies in the Optimized Adaptive Gamma Correction (OAGC) technique. The authors first generate an intermediate virtual image, similar to previous work [9], which aims to bridge the exposure gap. However, instead of directly fusing this virtual image, OAGC is applied to enhance it. OAGC works by:
- Defining an Objective Function: This function balances three key image quality metrics: contrast, saturation, and well-exposedness. It also includes a regularization term to prevent excessive deviation from the original virtual image.
- Optimizing Gamma Correction: The objective function is minimized to find an optimal gamma value ($\gamma^*$). This is achieved using an iterative gradient descent approach, as a closed-form solution is not available.
- Applying Gamma Correction: The optimized gamma value is then applied to the intermediate virtual image, significantly improving its contrast, saturation, and well-exposedness.
- Multi-Scale Fusion: Finally, the enhanced virtual image is fused with the original two input images using a standard multi-scale fusion approach (based on Gaussian and Laplacian pyramids), similar to methods like Mertens et al. [6].
The novelty lies in the OAGC method, which adaptively adjusts gamma correction based on image content to maximize desired quality metrics, thereby creating a much more informative intermediate image for fusion.
Datasets
The authors used the SICE dataset [14] for their experiments, which consists of 116 image pairs with various scenes and exposure differences. This dataset is suitable for evaluating both multi-exposure fusion and single-image enhancement tasks.
Key Insights and Ideas
- OAGC for Enhanced Virtual Images: The key insight is that simply interpolating between two exposures isn't enough when both are poorly exposed. Enhancing the intermediate virtual image before fusion is crucial for recovering lost details.
- Balancing Quality Metrics: The objective function in OAGC effectively balances contrast, saturation, and well-exposedness. This is important because optimizing for one metric might negatively impact others. The grid search and gradient descent results show that the optimal gamma strikes a good balance.
- Adaptability to Single Image Enhancement: The OAGC technique is not limited to fusion. It can be directly applied to a single image to improve its quality, demonstrating its versatility.
- Failure Case Analysis: The paper acknowledges that OAGC, like any method, cannot create content from nothing. If input images are extremely underexposed and lack any discernible features in certain regions, the fusion will still be limited.
Key Results
- Qualitative Improvement: The proposed method consistently produced visually superior fusion results compared to state-of-the-art methods like Kou's [8], DeepFuse [10], Yang's [9], and U2Fusion [13], especially in challenging cases with small exposure differences or when both inputs were underexposed/overexposed. The fused images showed better detail visibility in dark and bright regions.
- Quantitative Performance: In quantitative assessments using metrics like NIQE, BRISQUE, NR-IQA, and DE, the proposed method achieved the best scores across all categories. Specifically, it achieved the lowest NIQE and BRISQUE (indicating better naturalness and fewer distortions) and the highest DE (indicating better detail preservation).
- Single Image Enhancement: The method also demonstrated strong performance in single image enhancement, revealing unseen details and increasing color vividness more effectively than other compared methods.
Limitations and Future Work
- Content Creation Limitation: The paper explicitly states that OAGC cannot generate content where none exists. If both input images are extremely underexposed and lack information in specific regions, the fused output will also lack detail in those areas. Figure 9 illustrates a failure case where input images are very underexposed.
- Future Work: While not explicitly detailed, the success in single image enhancement suggests potential for further exploration in this area. The method's adaptability to different exposure ratios also opens avenues for more robust HDR imaging pipelines.
Summary updated