Abstract
Single image brightening is a challenging task due to the complex nature of noise, color distortion, and lightness preservation in low-light conditions. This paper proposes a hybrid learning framework that combines model-driven virtual image generation with a data-driven residual convolutional neural network (CNN). By leveraging Camera Response Functions (CRFs) to generate initial brightened versions and a CNN to refine details and suppress noise, the method achieves state-of-the-art results. The final output is produced via multi-scale exposure fusion, ensuring high structural similarity and natural color reproduction.
Introduction
Capturing high-quality images in low-light environments is a fundamental challenge in computer vision. Traditional solutions often involve long exposure times, which lead to motion blur, or high ISO settings, which introduce significant sensor noise. Digital post-processing to brighten under-exposed images is a viable alternative but frequently suffers from four major issues: noise amplification, highlight washout, lightness distortion, and color shift. This paper introduces a hybrid learning approach designed to mitigate these issues by integrating physical camera models with deep learning architectures.
Proposed Methodology
The proposed framework operates in three distinct stages, leveraging the strengths of both mathematical modeling and neural networks:
- Model-Driven Virtual Image Generation: Instead of simple linear scaling, the system utilizes Intensity Mapping Functions (IMFs) based on the specific Camera Response Function (CRF). This process generates two virtual images—one moderately bright and one highly bright—preserving the physical relationship between pixels and avoiding lightness distortion.
- Data-Driven Residual Enhancement: A Residual CNN is employed to learn the "unmodelled information," specifically the gap between the mathematically predicted images and the ground truth. This network utilizes a weighted restoration loss to handle noise and a specialized Color Loss (measuring the angle between RGB vectors) to maintain chromatic accuracy.
- Multi-Scale Exposure Fusion (MEF): The original dark image and the two enhanced virtual images are fused using a multi-scale approach. Custom weighting schemes ensure that the final result retains the sharp details of the original while adopting the visibility of the enhanced versions without washing out highlights.
Experimental Setup and Dataset
To train and evaluate the model, the authors curated a custom dataset consisting of 300 sets of image triplets (low, medium, and high exposure) captured with a Nikon 7200 camera at a fixed ISO of 800. Controlled conditions ensured perfect alignment between the low-light inputs and the high-exposure ground truths, providing a reliable baseline for training the residual network.
Results and Discussion
The hybrid method was evaluated against several state-of-the-art algorithms, including LIME, RetinexNet, and DeepUPE. Quantitative assessment using the MEF-SSIM metric, which evaluates structural and contrast preservation, placed the proposed method at 0.9368, significantly outperforming the next best competitor, DeepUPE (0.8637).
Qualitatively, the results demonstrate superior color preservation in shadow regions and a marked reduction in artifacts like halos or over-exposure in bright areas. The hybrid nature of the model also allows for faster convergence during training compared to purely data-driven networks because the model-driven component provides a strong initial estimate.
Conclusion
This research demonstrates that combining physical camera models with deep learning provides a robust solution for single image brightening. By delegating the primary brightening task to a model-driven stage and the refinement task to a CNN, the system achieves high-fidelity results with less training data. While the current implementation depends on known CRFs, future work will explore automatic CRF estimation to enhance the system's versatility across different imaging devices.
Summary updated