Luminance-aware Color Transform for Multiple Exposure Correction

Jong-Hyeon Baek, Daehyun Kim, Su-Min Choi, Hyo-Jun Lee, Hanul Kim, Yeong Jun KohICCV 2023 · DOI

Introduction

Correcting images with improper exposure—whether they are "blown out" (overexposed) or "crushed" (underexposed)—is a balancing act between restoring color and preserving fine detail. Standard deep learning architectures often introduce blurring during this process. This paper introduces Luminance-aware Color Transform (LACT), a novel framework that uses a set of learned, luminance-driven transformation functions to precisely map pixels back to their ideal state without sacrificing sharpness.

The Challenge of Exposure Correction

Most existing exposure correction methods suffer from two main flaws. First, they often have a narrow focus, designed specifically for underexposure (low-light enhancement) while failing on overexposed regions. Second, they frequently result in a loss of detail. Popular architectures like U-Nets rely on down-sampling and up-sampling, which can discard high-frequency textures, leading to "mushy" results.

Conceptual comparison between standard single-transform methods and the proposed multi-transform approach.

Simple global transformation functions, such as a single gamma curve, are often insufficient for complex scenes where different regions require different types of correction. The authors propose moving from a single transform to a set of transforms to handle these complexities.

Methodology: The LACT Framework

The LACT framework is designed to transform existing pixel intensities rather than generating new ones from a bottleneck, preserving structural integrity.

The complete LACT architecture, showing the Backbone, Luminance Comparison Module, and the color transform pipeline.

The framework consists of four primary components:

  1. Backbone Network: Extracts initial features using convolution and inverted residual blocks.
  2. Luminance Comparison Module (LCM): To help the network understand exposure, the authors use order learning. During training, the module compares two randomly sampled images to determine which is brighter, forcing the encoder to extract exposure-sensitive "luminance features."
  3. Luminance-aware Color Transform (LACT) Module: This module predicts transformation functions rather than pixels. It uses Multi-Head Cross-Attention (MCA) to integrate luminance features and generates 36 transformation functions for each color channel.
  4. Post-processing Module (PPM): The multi-channel representation is projected back to RGB and refined to ensure visually pleasing results.

Experimental Evaluation

The model was evaluated on two major benchmarks: the ME (Multi-Exposure) Dataset, containing over 20,000 images, and the SICE Dataset, featuring 589 scenes with diverse lighting. These datasets allow for testing the model's ability to handle both exposure extremes and generalize across various conditions.

Key Results and Insights

LACT achieved state-of-the-art (SOTA) results on both benchmarks. On the ME dataset, it achieved an average PSNR of 23.57 and SSIM of 0.869, outperforming previous leaders like FECNet and ENC.

Qualitatively, LACT is significantly better at recovering natural color tones in saturated regions (like bright skies) without the "halo" artifacts or blurring seen in U-Net based competitors. The core insights driving this success are:

  • Luminance as a Prior: Explicitly training the network to recognize relative brightness provides an "internal compass" for correction.
  • The Power of Many Curves: Fusing 36 different transforms allows for complex, non-linear, and locally adaptive corrections.
  • Preservation via Transformation: By avoiding generative decoders, the model inherently preserves the "edges" and "grit" of the original photo.

Limitations and Future Directions

While effective, the computational complexity of generating and fusing 36 transforms per channel may be higher than simpler models, and inference latency on mobile hardware remains to be detailed. Additionally, in cases of total "clipping" (pure white or black), the model must rely on surrounding context to estimate missing data. Future work could extend this luminance-aware approach to HDR reconstruction or video stabilization.

Summary updated

BibTeX
@inproceedings{baek2023luminance,
  author = {Jong-Hyeon Baek and Daehyun Kim and Su-Min Choi and Hyo-Jun Lee and Hanul Kim and Yeong Jun Koh},
  title = {{Luminance-aware Color Transform for Multiple Exposure Correction}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2023},
  doi = {10.1109/iccv51070.2023.00566},
}
Figure