Single Image Brightening via Multi-Scale Exposure Fusion with Hybrid Learning

Chaobing Zheng, Zhengguo Li, Yi Yang, Shiqian Wu

Abstract

Single image brightening is a challenging task due to the complex nature of noise, color distortion, and lightness preservation in low-light conditions. This paper proposes a hybrid learning framework that combines model-driven virtual image generation with a data-driven residual convolutional neural network (CNN). By leveraging Camera Response Functions (CRFs) to generate initial brightened versions and a CNN to refine details and suppress noise, the method achieves state-of-the-art results. The final output is produced via multi-scale exposure fusion, ensuring high structural similarity and natural color reproduction.

Introduction

Capturing high-quality images in low-light environments is a fundamental challenge in computer vision. Traditional solutions often involve long exposure times, which lead to motion blur, or high ISO settings, which introduce significant sensor noise. Digital post-processing to brighten under-exposed images is a viable alternative but frequently suffers from four major issues: noise amplification, highlight washout, lightness distortion, and color shift. This paper introduces a hybrid learning approach designed to mitigate these issues by integrating physical camera models with deep learning architectures.

Proposed Methodology

The proposed framework operates in three distinct stages, leveraging the strengths of both mathematical modeling and neural networks:

  1. Model-Driven Virtual Image Generation: Instead of simple linear scaling, the system utilizes Intensity Mapping Functions (IMFs) based on the specific Camera Response Function (CRF). This process generates two virtual images—one moderately bright and one highly bright—preserving the physical relationship between pixels and avoiding lightness distortion.
  2. Data-Driven Residual Enhancement: A Residual CNN is employed to learn the "unmodelled information," specifically the gap between the mathematically predicted images and the ground truth. This network utilizes a weighted restoration loss to handle noise and a specialized Color Loss (measuring the angle between RGB vectors) to maintain chromatic accuracy.
  3. Multi-Scale Exposure Fusion (MEF): The original dark image and the two enhanced virtual images are fused using a multi-scale approach. Custom weighting schemes ensure that the final result retains the sharp details of the original while adopting the visibility of the enhanced versions without washing out highlights.
Figure 1: The proposed hybrid learning framework, illustrating the integration of model-driven virtual image generation and data-driven CNN refinement.

Experimental Setup and Dataset

To train and evaluate the model, the authors curated a custom dataset consisting of 300 sets of image triplets (low, medium, and high exposure) captured with a Nikon 7200 camera at a fixed ISO of 800. Controlled conditions ensured perfect alignment between the low-light inputs and the high-exposure ground truths, providing a reliable baseline for training the residual network.

Results and Discussion

The hybrid method was evaluated against several state-of-the-art algorithms, including LIME, RetinexNet, and DeepUPE. Quantitative assessment using the MEF-SSIM metric, which evaluates structural and contrast preservation, placed the proposed method at 0.9368, significantly outperforming the next best competitor, DeepUPE (0.8637).

Figure 4: Qualitative comparison of the proposed method against existing techniques. Red circles highlight the superior detail and texture recovery achieved by the hybrid approach.

Qualitatively, the results demonstrate superior color preservation in shadow regions and a marked reduction in artifacts like halos or over-exposure in bright areas. The hybrid nature of the model also allows for faster convergence during training compared to purely data-driven networks because the model-driven component provides a strong initial estimate.

Conclusion

This research demonstrates that combining physical camera models with deep learning provides a robust solution for single image brightening. By delegating the primary brightening task to a model-driven stage and the refinement task to a CNN, the system achieves high-fidelity results with less training data. While the current implementation depends on known CRFs, future work will explore automatic CRF estimation to enhance the system's versatility across different imaging devices.

Summary updated

Abstract

A small ISO and a small exposure time are usually used to capture an image in the back or low light conditions which results in an image with negligible motion blur and small noise but look dark. In this paper, a single image brightening algorithm is introduced to brighten such an image. The proposed algorithm includes a unique hybrid learning framework to generate two virtual images with large exposure times. The virtual images are first generated via intensity mapping functions (IMFs) which are computed using camera response functions (CRFs) and this is a model-driven approach. Both the virtual images are then enhanced by using a data-driven approach, i.e. a residual convolutional neural network to approach the ground truth images. The model-driven approach and the data-driven one compensate each other in the proposed hybrid learning framework. The final brightened image is obtained by fusing the original image and two virtual images via a multi-scale exposure fusion algorithm with properly defined weights. Experimental results show that the proposed brightening algorithm outperforms existing algorithms in terms of the MEF-SSIM metric.
BibTeX
@article{zheng2021single,
  author = {Chaobing Zheng and Zhengguo Li and Yi Yang and Shiqian Wu},
  title = {{Single Image Brightening via Multi-Scale Exposure Fusion with Hybrid Learning}},
  journal = {IEEE Trans. Circuits Syst. Video Technol.},
  year = {2021},
  url = {https://openreview.net/forum?id=M9aCOePSbI},
}
Figure