LUT-GCE: Lookup Table Global Curve Estimation for Fast Low-light Image Enhancement

Jinhui Tang (唐金辉)arXiv 2023 · DOI · arXiv

TL;DR

This paper introduces LUT-GCE, a novel method for fast low-light image enhancement that uses a global cubic curve estimation network (GCENet) and a lookup table for efficient inference. By avoiding pixel-wise adjustments and leveraging a compact network with a lookup table, LUT-GCE achieves state-of-the-art speed while maintaining competitive image quality, making it suitable for real-time applications, even on high-resolution images.

What Problem Does This Paper Address?

Capturing clear images in low-light conditions is a persistent challenge across various applications, from smartphone photography to surveillance and autonomous driving. Traditional methods struggle with the trade-off between image quality and processing speed, especially as image resolutions increase (e.g., 1080p and 4K). Existing deep learning approaches often require significant computational resources and can lack generalization capabilities, particularly when trained on limited paired data. Zero-shot learning methods have improved generalization but often rely on pixel-wise curve adjustments, which become computationally prohibitive for high-resolution images. This paper aims to bridge this gap by developing a method that is both highly effective in enhancing low-light images and exceptionally fast, even for ultra-high-definition content.

Methods

The core of LUT-GCE lies in its innovative approach to curve estimation and inference. Instead of the quadratic curves used in prior work, the authors propose a more flexible cubic curve formulation for light enhancement. This cubic curve is designed to handle both under- and over-exposed regions simultaneously and, crucially, is applied globally to the entire image, eliminating the need for pixel-wise computations.

To estimate the parameters of this global cubic curve, they introduce a Global Curve Estimation Network (GCENet). This network is remarkably lightweight, boasting only 25.4k parameters. To further optimize performance, GCENet resizes input images to a fixed, smaller resolution (128x128) before processing, significantly reducing computational load without sacrificing effectiveness.

To achieve near-instantaneous inference, LUT-GCE employs a lookup table (LUT). The entire enhancement process, including preprocessing, curve application, and postprocessing, is precomputed for all possible pixel intensity values and stored in this LUT. This allows for extremely rapid retrieval of enhanced pixel values during inference, bypassing the need for repeated network computations.

Additionally, a novel histogram smoothness loss is introduced. This loss function encourages a more uniform distribution of pixel intensities in the enhanced image, which helps to improve contrast and recover finer details, contributing to better image quality.

Datasets

The paper evaluates LUT-GCE on several standard low-light image enhancement benchmarks:

  • LOL (Low-Light Image Enhancement) dataset [32]: A widely used dataset for low-light image enhancement.
  • VE-LOL (Very Low-Light Image Enhancement) dataset [13]: Focuses on extremely low-light conditions.
  • NPE (No-Reference Image Quality Evaluation) [28]: Used for no-reference performance evaluation.
  • LIME (Low-Light Image Enhancement) [5]: Another benchmark for low-light enhancement.
  • MEF (Multi-Exposure Fusion) [18]: Used for evaluating enhancement quality.
  • DICM (Dark Image Contrast Measurement) [10]: A dataset for contrast enhancement.
  • VV (VV dataset): Used for no-reference performance evaluation.
  • UHD-LOL4K [30]: A dataset specifically for ultra-high-definition low-light images.
  • ACDC [26]: Used for evaluating performance on low-light semantic segmentation.
  • EXDark [15]: Used for evaluating performance on low-light object detection.

The training of GCENet was performed on the SICE dataset [2], specifically using multi-exposure sequences.

Key Insights and Ideas

  • Global Curve for Efficiency: The shift from pixel-wise curve adjustments to a single, global cubic curve is a key enabler of speed. This global approach dramatically reduces computational complexity.
  • Cubic Curve Versatility: The proposed cubic curve formulation is more expressive than quadratic curves, allowing it to effectively handle both under- and over-exposed regions within a single image without complex pixel-level logic.
  • Lookup Table for Real-Time Inference: Precomputing the entire enhancement pipeline into a lookup table is a powerful technique for achieving near-instantaneous results, making the method practical for real-time applications.
  • Lightweight Network Design: The GCENet's minimal parameter count (25.4k) is crucial for its efficiency, demonstrating that high performance doesn't always require massive models.
  • Histogram Smoothness Loss: This novel loss function is effective in improving image contrast and detail recovery, which are critical aspects of low-light enhancement, especially in a zero-shot learning context.

Key Results

  • Superior Inference Speed: LUT-GCE achieves significantly faster inference times compared to other state-of-the-art methods, particularly on high-resolution images (e.g., 4K). For 4K images, it is reported to be up to 5 times faster than other millisecond-level methods. The lookup table inference is especially rapid, costing only 1/10 of Zero-DCE++ and 1/7 of SCI for 4K images.
  • Competitive Image Quality: Despite its speed, LUT-GCE achieves competitive results on various full-reference and no-reference metrics, often outperforming unsupervised methods and performing comparably to supervised ones. It shows superior performance in the perceptual metric LPIPS.
  • Effective for High-Level Tasks: The method demonstrates its utility by improving performance in downstream tasks like semantic segmentation and object detection on low-light images.

Limitations and Future Work

  • Resizing Impact: While resizing to 128x128 significantly boosts efficiency, the paper acknowledges that this is a trade-off between effectiveness and efficiency. Further research could explore adaptive resizing strategies or more sophisticated downsampling techniques.
  • Zero-Shot Learning Constraints: The method is designed for zero-shot learning, meaning it doesn't require paired data. However, like many zero-shot methods, its performance might be further improved with fine-tuning on specific datasets or domains.
  • Future Work: The authors suggest applying their method to high-level vision tasks, which they have already begun to explore. Further integration with other vision pipelines and exploring more complex curve formulations or network architectures could be future directions.

Summary updated

Abstract

We present an effective and efficient approach for low-light image enhancement, named Lookup Table Global Curve Estimation (LUT-GCE). In contrast to existing curve-based methods with pixel-wise adjustment, we propose to estimate a global curve for the entire image that allows corrections for both under- and over-exposure. Specifically, we develop a novel cubic curve formulation for light enhancement, which enables an image-adaptive and pixel-independent curve for the range adjustment of an image. We then propose a global curve estimation network (GCENet), a very light network with only 25.4k parameters. To further speed up the inference speed, a lookup table method is employed for fast retrieval. In addition, a novel histogram smoothness loss is designed to enable zero-shot learning, which is able to improve the contrast of the image and recover clearer details. Quantitative and qualitative results demonstrate the effectiveness of the proposed approach. Furthermore, our approach outperforms the state of the art in terms of inference speed, especially on high-definition images (e.g., 1080p and 4k).
BibTeX
@misc{2023lut,
  author = {Jinhui Tang (唐金辉)},
  title = {{LUT-GCE: Lookup Table Global Curve Estimation for Fast Low-light Image Enhancement}},
  year = {2023},
  eprint = {2306.07083},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2306.07083},
}
Figure