Fast, Zero-Reference Low-Light Image Enhancement with Camera Response Model

Xiaofeng Wang, Liang Huang, Mingxuan Li, Chengshan Han, Xin Liu, Ting NieSensors 2024 · DOI

TL;DR

This paper introduces ZRCRN, a novel deep learning method for fast, zero-reference low-light image enhancement. By leveraging a camera response model and a streamlined network, it achieves high-quality results in a single pass, significantly outperforming existing methods in speed while maintaining comparable accuracy.

What Problem Does This Paper Address?

Low-light images are a common challenge in various applications like surveillance, autonomous driving, and remote sensing. Their inherent low brightness, poor contrast, and noise make them difficult for subsequent analysis. While existing low-light image enhancement (LLIE) techniques exist, many suffer from complex network architectures, require extensive training data (paired images), or involve multiple iterative steps, hindering their real-time applicability and generalization. This paper aims to bridge this gap by developing an efficient, reference-free LLIE method that can enhance images quickly and effectively.

Methods

The core of the proposed method, ZRCRN (Zero-Reference Camera Response Network), is built upon a Camera Response Model (CRM). The process involves:

  1. Inverse Camera Response Function (CRF) Transformation: The input low-light image ($P_0$) is first transformed using an inverse CRF to obtain a radiation map ($E_0$). This step simplifies the subsequent parameter extraction by reducing non-linearity.
  2. Streamlined Parameter-Generating Network (PGN): A double-layer convolutional network is employed to extract a single-channel, pixel-wise exposure ratio map ($K$) from $E_0$. This PGN is designed to be efficient, using a limited number of layers and channels.
  3. Brightness Transformation Function (BTF): The extracted exposure ratio map ($K$) is then used by the BTF to transform the original low-light image ($P_0$) into an enhanced image ($P_1$) in a single step.

To train this network without paired data, two novel loss functions are introduced:

  • Contrast-Preserving Brightness Loss: This loss function generates a target brightness map by linearly stretching a derived brightness map from the input image. This helps enhance brightness and contrast while preserving the original spatial distribution of brightness.
  • Edge-Preserving Smoothness Loss: This loss function aims to reduce noise and enhance details. It involves edge detection, gradient extraction, and selective scaling to amplify gradients at edges while suppressing noise in non-edge regions.

Datasets

The paper utilizes the following datasets:

  • Training: SICE dataset (multi-exposure image sequences).
  • Evaluation (Full-Reference - FR): LOL dataset, LOL_v2 dataset.
  • Evaluation (No-Reference - NR): DICM dataset, LIME dataset, MEF dataset.
  • Downstream Task Evaluation: DARK FACE dataset (for night face detection).

Key Insights and Ideas

  • Camera Response Model as a Foundation: Leveraging the CRM provides a principled way to model the image formation process, allowing for efficient extraction of essential parameters like the exposure ratio.
  • Inverse Transformation for Simplification: Inverting the CRF before parameter extraction significantly simplifies the PGN, leading to higher efficiency and better performance.
  • Reference-Free Loss Functions are Crucial: The development of contrast-preserving brightness loss and edge-preserving smoothness loss is key to achieving high-quality enhancement without relying on ground truth images, making the method more versatile.
  • Single-Pass Enhancement for Speed: The architecture is designed for a single transformation step, drastically reducing processing time compared to iterative methods.
  • Balancing Speed and Quality: The paper demonstrates that high-speed enhancement doesn't necessarily mean sacrificing image quality, achieving competitive results with state-of-the-art methods.

Key Results

  • Speed: ZRCRN achieves processing speeds more than twice as fast as similar methods like Zero_DCE and EnlightenGAN.
  • Accuracy: The method achieves comparable SSIM and PSNR scores to other state-of-the-art LLIE methods on both FR and NR datasets. For instance, on LOL_v2, it achieved an SSIM of 0.69 and PSNR of 18.20.
  • Generalization: ZRCRN shows excellent generalization ability on various datasets, outperforming supervised methods on NR datasets.
  • Downstream Task Performance: In night face detection, ZRCRN-enhanced images led to improved detection accuracy (AP of 0.451) compared to other enhancement methods, demonstrating its practical utility.
  • Ablation Studies: The PGN with $m=2$ and $n=2$ layers/channels was found to be optimal for balancing accuracy and efficiency. The parameter $a=0.2$ for contrast-preserving brightness loss and $\alpha=0.707, \beta=1.414, \gamma=2$ for edge-preserving smoothness loss were identified as optimal.

Limitations and Future Work

  • Precision in Object Detection: While ZRCRN significantly improves detection performance, the paper acknowledges that the precision is "still not very good."
  • Future Work: The authors suggest that more diverse training data could help improve precision. They also plan to explore further applications in object detection and aim to address the precision limitations in future work.

Summary updated

Abstract

Low-light images are prevalent in intelligent monitoring and many other applications, with low brightness hindering further processing. Although low-light image enhancement can reduce the influence of such problems, current methods often involve a complex network structure or many iterations, which are not conducive to their efficiency. This paper proposes a Zero-Reference Camera Response Network using a camera response model to achieve efficient enhancement for arbitrary low-light images. A double-layer parameter-generating network with a streamlined structure is established to extract the exposure ratio K from the radiation map, which is obtained by inverting the input through a camera response function. Then, K is used as the parameter of a brightness transformation function for one transformation on the low-light image to realize enhancement. In addition, a contrast-preserving brightness loss and an edge-preserving smoothness loss are designed without the requirement for references from the dataset. Both can further retain some key information in the inputs to improve precision. The enhancement is simplified and can reach more than twice the speed of similar methods. Extensive experiments on several LLIE datasets and the DARK FACE face detection dataset fully demonstrate our method's advantages, both subjectively and objectively.
BibTeX
@inproceedings{wang2024fast,
  author = {Xiaofeng Wang and Liang Huang and Mingxuan Li and Chengshan Han and Xin Liu and Ting Nie},
  title = {{Fast, Zero-Reference Low-Light Image Enhancement with Camera Response Model}},
  booktitle = {Sensors},
  year = {2024},
  doi = {10.3390/s24155019},
}
Figure