Representative Color Transform for Image Enhancement

Hanul Kim, Su-Min Choi, Chang-Su Kim, Yeong Jun KohICCV 2021 · DOI

Introduction

Image enhancement is a critical task in computer vision, aiming to improve the visual quality of photos by adjusting exposure, color, and contrast. Traditionally, this has been approached through two distinct methodologies: deep encoder-decoder models and global intensity transformations. While encoder-decoder models (like U-Nets) excel at understanding spatial context, they often suffer from detail loss due to down-sampling and up-sampling operations. Conversely, global intensity transforms, such as 3D Look-Up Tables (LUTs), preserve pixel-level details but lack the capacity to handle complex, spatially-varying adjustments.

To bridge this gap, researchers have introduced RCTNet, a framework built upon the Representative Color Transform (RCT). This approach combines the high-level contextual awareness of deep learning with the precision of color-mapping functions.

The Challenge of Image Enhancement

The fundamental trade-off in image enhancement is between structural preservation and contextual flexibility. Encoder-decoder architectures often "smudge" fine textures, while global transforms cannot distinguish between different regions of an image (e.g., brightening a foreground subject without overexposing the sky).

Comparison of image enhancement paradigms: (a) Encoder-Decoder, (b) Intensity Transform, and (c) the proposed Representative Color Transform.

As shown in the figure above, the Representative Color Transform (c) identifies a discrete set of colors that represent the image and learns how to transform them, allowing for sophisticated adjustments that remain faithful to the original image's structure.

Methodology: The RCTNet Architecture

The core innovation of RCTNet is the decoupling of color transformation from image structure. Instead of predicting pixel values directly, the network estimates a transformation for a small set of "representative colors."

The Three-Step Process

  1. Identification: An encoder analyzes the input to select an image-specific color palette (representative colors).
  2. Transformation: The model predicts the target values for these palette colors in the enhanced domain.
  3. Mapping (Attention): A scaled-dot product attention mechanism calculates the similarity between every pixel in the original image and the representative colors. The final image is reconstructed by blending the transformed colors based on these similarity weights.

System Components

RCTNet utilizes a multi-scale approach to ensure both global consistency and local detail:

  • Global RCT: Manages image-wide adjustments like overall exposure and white balance.
  • Local RCT: Employs a 31x31 mesh grid to handle regional variations, allowing the model to apply different enhancements to different parts of the scene.
  • Feature Fusion: This module integrates information from various scales of the encoder to inform the transformation process.
Detailed architecture of RCTNet, illustrating the multi-scale encoder, feature fusion module, and the dual Global/Local RCT branches.

Experimental Results

RCTNet was evaluated across several demanding benchmarks, including MIT-Adobe 5K for photo retouching, the LoL dataset for low-light enhancement, and the EUVP/UIEB datasets for underwater imaging.

In all categories, RCTNet achieved state-of-the-art (SOTA) performance:

  • MIT-Adobe 5K: Achieved a PSNR of 26.02, surpassing previous leaders like 3D LUT (25.21).
  • Low-Light (LoL): Reached a PSNR of 22.67, significantly outperforming specialized models like KinD (20.87).
  • Underwater Imaging: On the EUVP dataset, it achieved a PSNR of 26.43, a substantial improvement over the previous best of 24.21.

Qualitatively, RCTNet produces images with vibrant, natural colors while maintaining sharp edges and fine architectural details that are often blurred by competing deep learning models.

Conclusion and Future Work

By treating image enhancement as a learnable color transformation rather than a direct pixel generation task, RCTNet achieves a unique balance of detail preservation and contextual intelligence. Its resolution-independent inference makes it particularly suitable for high-resolution photography. While the current model is less effective at denoising heavy sensor noise in extreme low-light conditions, future work aims to integrate sophisticated denoising modules directly into the RCT pipeline to create a truly comprehensive enhancement solution.

Summary updated

BibTeX
@inproceedings{kim2021representative,
  author = {Hanul Kim and Su-Min Choi and Chang-Su Kim and Yeong Jun Koh},
  title = {{Representative Color Transform for Image Enhancement}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2021},
  doi = {10.1109/iccv48922.2021.00442},
}
Figure