Abstract
Capturing photographs with wrong exposures remains a major source of errors in camera-based imaging. Exposure problems are categorized as either: (i) overexposed, where the camera exposure was too long, resulting in bright and washed-out image regions, or (ii) underexposed, where the exposure was too short, resulting in dark regions. Both under- and overexposure greatly reduce the contrast and visual appeal of an image. Prior work mainly focuses on underexposed images or general image enhancement. In contrast, our proposed method targets both over- and under-exposure errors in photographs. We formulate the exposure correction problem as two main sub-problems: (i) color enhancement and (ii) detail enhancement. Accordingly, we propose a coarse-to-fine deep neural network (DNN) model, trainable in an end-to-end manner, that addresses each sub-problem separately. A key aspect of our solution is a new dataset of over 24,000 images exhibiting the broadest range of exposure values to date with a corresponding properly exposed image. Our method achieves results on par with existing state-of-the-art methods on underexposed images and yields significant improvements for images suffering from overexposure errors.
The Challenge of Exposure Correction
Exposure errors are a fundamental hurdle in digital photography. While modern cameras attempt to automate exposure settings, they frequently struggle in challenging lighting conditions, such as high-contrast scenes or strong backlighting. These errors manifest in two ways: underexposure, where details are lost in shadows ("crushed"), and overexposure, where bright regions become "washed out" or "clipped."
Overexposure is particularly difficult to rectify because it often involves a loss of color saturation and structural detail. Furthermore, camera Image Signal Processors (ISPs) apply non-linear operations to render final 8-bit sRGB images, making the reversal of these errors mathematically complex. This research introduces a unified deep learning model designed to correct the full spectrum of exposure errors in the standard sRGB color space.
Multi-Scale Deep Learning Architecture
The proposed solution utilizes a coarse-to-fine deep neural network (DNN) architecture that mimics manual photo editing by first addressing global color and then refining local details.
- Laplacian Pyramid Decomposition: The input image is decomposed into multiple frequency bands. The lowest level captures global color information, while higher levels contain increasingly fine textures.
- Sequential Sub-networks: The model employs a series of U-Net style sub-networks. The primary sub-network processes the lowest frequency level for global color correction. Its output is then upsampled and passed to subsequent sub-networks that integrate corrected details from higher pyramid levels.
- Multi-Scale Losses: The training process uses a combination of $L_1$ reconstruction loss, pyramid loss (to guide intermediate sub-networks), and adversarial loss to ensure perceptually realistic results.
- Bilateral Guided Upsampling: To maintain efficiency on high-resolution images (e.g., 16 megapixels), the model processes a downsampled version and uses a bilateral grid to transfer corrections back to the original resolution, preventing GPU memory exhaustion.
A Comprehensive Exposure Dataset
A significant contribution of this work is the creation of the Exposure Correction Dataset, which is broader in scope than previous benchmarks.
Derived from the MIT-Adobe FiveK dataset, it contains 24,330 sRGB images. Each scene is rendered at five different relative exposure values (-1.5, -1, 0, +1, and +1.5 EV) using the Adobe Camera Raw SDK to accurately emulate real camera ISP behavior. Crucially, the ground truth targets are high-quality versions manually retouched by expert photographers, allowing the model to learn aesthetic preferences rather than just standard auto-exposure.
Key Results and Insights
- Quantitative Superiority: The method outperforms existing state-of-the-art models like Deep UPE and DPED across PSNR, SSIM, and Perceptual Index (PI) metrics, with particularly notable gains in overexposed image recovery.
- Qualitative Quality: The model produces images with natural color reproduction and minimal artifacts, such as halos or color bleeding, compared to traditional histogram equalization or HDR tools.
- Efficiency: The bilateral guided upsampling allows the system to process a 16-megapixel image in approximately 4.5 seconds, making it viable for real-world deployment.
- Recoverability of Overexposure: The study demonstrates that even "washed-out" sRGB regions often retain enough residual information in a multi-scale representation for a DNN to reconstruct plausible colors and textures.
Limitations and Future Directions
Despite its success, the model faces challenges with "semantic blindness." In cases of total clipping (pure white with zero texture), the lack of semantic context can lead to color bleeding. Future work could integrate semantic segmentation to help the model "understand" the objects it is correcting. Additionally, extreme underexposure can sometimes lead to amplified sensor noise. The authors envision this dataset and multi-scale approach serving as a foundation for broader image enhancement tasks, including white balance correction.
Summary updated