UIEC^2-Net: CNN-based Underwater Image Enhancement Using Two Color Space

Yudong Wang, Jichang Guo, Huan Gao, Huihui Yue

TL;DR

UIEC^2-Net is the first deep learning-based underwater image enhancement framework to integrate both RGB and HSV color spaces. By combining local pixel-level RGB corrections with global HSV adjustments via a novel neural curve layer, it effectively resolves color casts, low contrast, and under-saturation in a lightweight, end-to-end trainable network.

What Problem Does This Paper Address?

Capturing clear images underwater is notoriously difficult. Due to light absorption and scattering, raw underwater images suffer from severe quality degradation: red light disappears first, causing a heavy bluish or greenish color cast, while backscattering leads to low contrast, dim lighting, and blurry details.

While Convolutional Neural Networks (CNNs) and Generative Adversarial Networks (GANs) have advanced underwater image enhancement, almost all existing deep learning models operate exclusively within the RGB color space. The problem? RGB channels are highly correlated and don't directly reflect human-perceived image properties like luminance (brightness) and saturation. Consequently, RGB-only networks struggle to adjust global contrast and brightness effectively without introducing artifacts or overexposure.

Figure 1: Samples of real-world underwater image enhance- ment by UIECˆ2-Net. Top row: raw underwater images from DUIE datasets [1]; Bottom row: the corresponding results of our model.

Methods

The authors introduce UIEC^2-Net (Underwater Image Enhancement CNN using 2 Color Spaces), an end-to-end architecture composed of three distinct functional blocks:

Figure 2: An overview of the proposed UIECˆ2-Net architecture. UIECˆ2-Net consists of three blocks trained end-to-end: a RGB pixel-level block for simple and basic processing, a HSV global-adjust block that leverages neural curve layers for globally refine image property(saturation and luminance), and a attention map block for getting better underwater enhanced image through attention mechanism.
  1. RGB Pixel-Level Block: A plain, fully convolutional network (8 layers, no downsampling or skip connections) designed for fundamental local operations like denoising and removing color casts.
  2. HSV Global-Adjust Block: The RGB output is converted into the HSV color space using a differentiable transformation layer. A small CNN then regresses the parameters of a neural curve layer—a piece-wise linear scaling curve that globally adjusts luminance and saturation.
  3. Attention Map Block: Because the Hue (H) channel in HSV is highly sensitive, direct HSV manipulation can sometimes cause color distortion. To prevent this, an attention block learns pixel-level weights to intelligently blend the strengths of both the RGB and HSV block outputs.

To enable backpropagation across the entire pipeline, the authors implemented fully differentiable RGB-to-HSV and HSV-to-RGB conversion formulas, replacing standard non-differentiable conditional operations with piece-wise linear functions.

The network is optimized using a multi-term loss function combining $L_1$ loss, SSIM loss, perceptual loss (via VGG-19), and a specialized HSV conical space loss to ensure color fidelity.

Datasets

The model was trained and evaluated on a mix of synthetic and real-world datasets (2,000 total training images):

  • Synthetic Data: 1,200 image pairs generated from the indoor NYU-v2 RGB-D dataset, simulating 10 distinct types of underwater optical degradation (turbidity, light attenuation, etc.).
  • Real-World Data: 800 pairs from the Underwater Image Enhancement Benchmark Dataset (UIEBD), which includes diverse real-world underwater scenes paired with high-quality reference images voted on by human volunteers.
  • Testing: Evaluated on 900 synthetic and 90 real-world underwater images.

Key Insights and Ideas

  • Color Space Synergy: Instead of forcing a single color space to do all the heavy lifting, the paper exploits the unique physics of two: RGB handles local pixel-level corrections (like removing a green tint), while HSV handles global statistical properties (like boosting contrast and vividness).
  • Curve Parameterization over Pixel Prediction: In the HSV block, the network doesn't predict pixel values directly. Instead, it predicts the shape of an enhancement curve (Figure 3). This regularizes the output, preventing extreme pixel values and avoiding the overexposure common in direct pixel-mapping networks.
Examples of learnt neural global adjustment curves in HSV color space.
  • Pixel-Level vs. Channel-Level Attention: While many networks use channel-level attention (e.g., SE-Net), UIEC^2-Net uses pixel-level attention maps. This allows the network to apply HSV-derived brightness boosts to dark background regions while relying on the stable RGB block for complex foreground textures.

Key Results

UIEC^2-Net significantly outperformed traditional methods (Histogram Equalization, White Balance, UDCP, ULAP) and deep learning baselines (UWGAN, UGAN, UWCNN, DUIENet) across both synthetic and real-world tests.

Quantitative Performance

On real-world datasets (UIEBD), UIEC^2-Net achieved top marks in both full-reference and no-reference metrics:

  • PSNR: 24.56 dB (compared to the next best, UGAN, at 21.30 dB).
  • SSIM: 0.9346 (compared to HE at 0.8509).
  • UCIQE / UIQM: Achieved the highest scores among deep learning methods, demonstrating superior color density, saturation, and contrast balance without artificial over-enhancement.

Qualitative Performance

Visually, the model effectively strips away the characteristic "underwater haze" and background color casts while preserving fine structural details and avoiding the noise amplification seen in traditional histogram-stretching techniques.

Limitations and Future Work

The authors note that the RGB pixel-level block currently relies on a relatively simple, plain fully convolutional network architecture. While it performs remarkably well, they acknowledge that integrating more advanced, widely adopted backbones—such as a U-Net architecture or deep residual networks (ResNets)—could further boost the network's capacity to handle highly intricate local textures and severe degradation.

Summary updated

Abstract

Underwater image enhancement has attracted much attention due to the rise of marine resource development in recent years. Benefit from the powerful representation capabilities of Convolution Neural Networks(CNNs), multiple underwater image enhancement algorithms based on CNNs have been proposed in the last few years. However, almost all of these algorithms employ RGB color space setting, which is insensitive to image properties such as luminance and saturation. To address this problem, we proposed Underwater Image Enhancement Convolution Neural Network using 2 Color Space (UICE^2-Net) that efficiently and effectively integrate both RGB Color Space and HSV Color Space in one single CNN. To our best knowledge, this method is the first to use HSV color space for underwater image enhancement based on deep learning. UIEC^2-Net is an end-to-end trainable network, consisting of three blocks as follow: a RGB pixel-level block implements fundamental operations such as denoising and removing color cast, a HSV global-adjust block for globally adjusting underwater image luminance, color and saturation by adopting a novel neural curve layer, and an attention map block for combining the advantages of RGB and HSV block output images by distributing weight to each pixel. Experimental results on synthetic and real-world underwater images show the good performance of our proposed method in both subjective comparisons and objective metrics. The code are available at https://github.com/BIGWangYuDong/UWEnhancement.
BibTeX
@inproceedings{wang2021uiec,
  author = {Yudong Wang and Jichang Guo and Huan Gao and Huihui Yue},
  title = {{UIEC^2-Net: CNN-based Underwater Image Enhancement Using Two Color Space}},
  booktitle = {Signal Processing: Image Communication},
  year = {2021},
  url = {https://openreview.net/forum?id=shqr6SpMLW},
}
Figure