Learning Color Representations for Low-Light Image Enhancement

Bomi Kim, Sunhyeok Lee, Nahyun Kim, Donggon Jang, Dae-Shik Kim

Abstract

Low-light image enhancement is a critical task in computer vision, yet most existing models struggle to recover authentic color distributions, often resulting in "washed out" or statistically inaccurate images. This paper introduces Channel-aware Color histogram Matching (CCM), a novel framework that prioritizes high-order color representations. By leveraging inter-channel dependencies through Layer Normalization and a differentiable color histogram loss based on Kernel Density Estimation, CCM restores vibrant and statistically accurate colors. Experimental results on synthetic and real-world datasets demonstrate that CCM achieves state-of-the-art performance in both visual quality and statistical color accuracy.

The Challenge of Color in the Dark

When capturing images in near-darkness, the problem extends beyond mere dimness; the underlying color information is fundamentally distorted. Pixel intensities shift, and the statistical distribution of colors becomes difficult to recover through simple brightening.

A typical low-light input image where color information is suppressed and distorted.

While deep learning has revolutionized image restoration, many models prioritize "high-level" visual information like edges and textures while neglecting high-order color representations. Standard normalization techniques often treat color channels independently, which destroys the subtle relationships between Red, Green, and Blue channels that define natural-looking color. The challenge lies in teaching a neural network to "learn" a complex color histogram.

Methodology: Channel-aware Color Histogram Matching

The authors propose a two-pronged approach to fix the "color-blindness" of typical low-light models:

1. $\gamma$-Channel Attention Residual Block ($\gamma$CRB)

Standard models often use Batch Normalization (BN), which normalizes each channel independently. The authors found BN to be ineffective for color restoration. Instead, they utilize Layer Normalization (LN), which computes statistics across all channels for a single pixel. They take the re-scaling factor ($\gamma$) from the LN layer and use it to weight a channel-attention mechanism, ensuring the model captures how channels depend on one another.

2. Differentiable Color Histogram Loss

Histograms are usually discrete and "spiky," making them impossible to use directly in gradient-based optimization. To solve this, the authors use Kernel Density Estimation (KDE) with a specific sigmoid-based kernel to create a smooth, differentiable approximation of the color histogram. This allows the network to be trained directly against the statistical distribution of the ground truth.

The ground-truth color histogram representing the target statistical distribution for the enhancement process.

3. Frequency Separation

To prevent the model from confusing color and texture, a Gaussian filter splits the image into low-frequency (color/shading) and high-frequency (edges/details) components. The histogram loss is applied only to the low-frequency part, while a Structural Similarity (SSIM) loss handles the edges.

Experimental Results

The researchers evaluated their method on two primary benchmarks: a synthetic dataset derived from RAISE and the real-world LOL (Low-Light) dataset.

The proposed CCM method achieved state-of-the-art (SOTA) results across multiple metrics:

  • PSNR/SSIM: On the real-world dataset, CCM reached a PSNR of 24.12 dB, significantly outperforming popular models like KinD (23.00 dB) and DRBN (21.27 dB).
  • Color Accuracy: Using the Wasserstein Distance (WD) to measure the statistical "distance" between predicted and ground-truth color histograms, CCM achieved the lowest scores (760.7 on real-world data), proving its output is statistically closer to the original colors.
  • Qualitative Quality: Visual results show fewer artifacts and more natural skin tones compared to Retinex-based or standard GAN approaches, which often produce "washed out" images.

Conclusion and Future Work

By shifting the focus from simple pixel-wise restoration to high-order color representations, CCM provides a robust solution for low-light enhancement. The integration of inter-channel awareness and differentiable histogram matching allows for the recovery of vibrant, natural-looking images. The authors acknowledge that the method currently relies on paired datasets, which are difficult to capture. Future directions include extending this work into semi-supervised or unsupervised learning to allow the model to learn color representations from unpaired "wild" images.

Summary updated

Abstract

Color conveys important information about the visible world. However, under low-light conditions, both pixel intensity, as well as true color distribution, can be significantly shifted. Moreover, most of such distortions are non-recoverable due to inverse problems. In the present study, we utilized recent advancements in learning-based methods for low-light image enhancement. However, while most "deep learning" methods aim to restore high-level and object-oriented visual information, we hypothesized that learning-based methods can also be used for restoring color-based information. To address this question, we propose a novel color representation learning method for low-light image enhancement. More specifically, we used a channel-aware residual network and a differentiable intensity histogram to capture color features. Experimental results using synthetic and natural datasets suggest that the proposed learning scheme achieves state-of-the-art performance. We conclude from our study that inter-channel dependency and color distribution matching are crucial factors for learning color representations under low-light conditions.
BibTeX
@inproceedings{kim2022learning,
  author = {Bomi Kim and Sunhyeok Lee and Nahyun Kim and Donggon Jang and Dae-Shik Kim},
  title = {{Learning Color Representations for Low-Light Image Enhancement}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2022},
  doi = {10.1109/wacv51458.2022.00098},
}
Figure