Abstract
Automatic color enhancement aims to adaptively adjust photos to achieve professional styles and tones. Current deep learning methods often struggle to simultaneously maintain global harmonious perception and fine-grained local details within a single model. To address this, we propose a coarse-to-fine framework utilizing non-local attention. Our approach divides the enhancement process into two cascaded stages: channel-wise enhancement and pixel-wise refinement. The first stage predicts a global linear mapping for RGB channels to perform broad style adjustments. The second stage learns a refining mapping via residual learning for local corrections. Crucially, we incorporate a non-local attention block to capture long-range dependencies, ensuring global consistency during local refinement. We evaluate our framework on standard benchmarks, demonstrating state-of-the-art performance and the effectiveness of each component.
Introduction
Automatic photo retouching is a challenging balancing act. A purely global adjustment, such as applying a single filter, preserves color harmony but often fails to address local issues like uneven lighting or specific object colors. Conversely, independent pixel-wise adjustments can introduce spatial artifacts—inconsistent colors or splotches—because the model lacks a global context of the image's "vibe."
Researchers from the University of Science and Technology of China (USTC) have developed a framework that mimics a professional editor's workflow: first setting the overall tone, then refining specific areas. This "coarse-to-fine" strategy ensures that the final output is both globally consistent and locally detailed.
Methodology
The proposed architecture consists of two primary cascaded networks: the Channel-wise Enhancement Network (CENet) and the Pixel-wise Refinement Network (PRNet).
Channel-wise Enhancement Network (CENet)
CENet handles the "coarse" enhancement step. It utilizes a ResNet50 backbone to extract high-level features, followed by global average pooling and fully connected layers. The network regresses a 12-dimensional vector representing a $3 \times 3$ weight matrix and a $3 \times 1$ bias vector. This defines a global linear mapping applied to the input image's RGB channels. By restricting CENet to only 12 parameters, the model is forced to focus on global style adjustment, preventing it from over-fitting to local details and ensuring spatial harmony.
Pixel-wise Refinement Network (PRNet)
The output of CENet serves as the input for PRNet, which performs the "fine" refinement. PRNet is designed as an encoder-decoder structure with three main stages:
- Downsampling: Three convolutional layers reduce the spatial resolution to extract compact features.
- Feature Transformation: This stage includes a Non-Local Attention Block and three residual blocks. The non-local block allows the network to capture long-range dependencies, enabling pixels in one part of the image (e.g., the left side of a sky) to "attend" to distant pixels (the right side of the sky), maintaining color consistency.
- Upsampling: Two deconvolutional layers and a final convolution with a Tanh activation generate a residual image ($r_p$).
The final retouched image is the sum of the coarse result and this learned local residual, making the learning task more stable and efficient.
Experimental Results
Dataset and Metrics
The framework was evaluated using the MIT-Adobe FiveK dataset, containing 5,000 raw images retouched by five professional photographers. Following standard protocols, the retouches from "Photographer C" were used as the ground truth. Performance was measured using $L^2$ error in CIELab color space, Peak Signal-to-Noise Ratio (PSNR), and Structural Similarity Index (SSIM).
Quantitative Analysis
Ablation studies confirmed the necessity of both stages. The full model (CE+PRNL) achieved an $L^2$ error of 9.10, significantly outperforming global-only (10.32) and local-only (10.93) configurations. Compared to established benchmarks, our model surpassed methods like DeepRL ($L^2$: 10.99) and DPE ($L^2$: 9.93), reaching a PSNR of 24.19 dB.
Qualitative Analysis
Visual comparisons demonstrate that the coarse-to-fine approach avoids the blurring and patchy artifacts common in purely pixel-wise methods. The non-local attention mechanism effectively prevents local adjustments from clashing with the overall image context, resulting in retouched photos that are remarkably close to professional ground truth.
Conclusion
This research presents a robust framework for automatic color enhancement by decoupling global style adjustment from local refinement. The integration of non-local attention provides a critical bridge between these two scales, ensuring consistency across the image. While the current model excels at standard resolutions, future work will explore more efficient attention mechanisms to handle high-resolution imagery and investigate unsupervised learning techniques to reduce reliance on expensive paired datasets.
Summary updated