Underwater Organism Color Enhancement via Color Code Decomposition, Adaptation and Interpolation

Xiaofeng Cong, Jing Zhang, Yeying Jin, Junming Hou, Yu Zhao, Jie Gui, James Tin-Yau Kwok, Yuan Yan TangarXiv 2024 · DOI · OpenReview

Abstract

Traditional underwater image enhancement (UIE) methods typically produce deterministic, "one-size-fits-all" outputs that lack flexibility for diverse aesthetic or scientific requirements. We present ColorCode, a novel framework that treats UIE as a decomposition and reconstruction task. By separating an image into distinct content and color codes and constraining the color latent space to a Gaussian distribution, ColorCode enables three powerful capabilities: standard color enhancement, color adaptation based on guidance images, and continuous color interpolation for realistic variations. Our approach achieves state-of-the-art performance across multiple benchmarks while providing unprecedented user control over the enhancement process.

Introduction

Underwater photography is fundamentally limited by the physics of light attenuation. As light travels through water, longer wavelengths (red and yellow) are absorbed more rapidly than shorter ones (blue and green), resulting in a pervasive color cast and loss of vibrant detail. While deep learning has advanced UIE, existing models are largely deterministic, offering a single fixed result for any given input. This lack of flexibility is a significant hurdle for researchers and artists who require specific color balances. Furthermore, many models fail to preserve the semantic integrity of colors because they do not explicitly separate image structure from color information.

Methods: The ColorCode Framework

The ColorCode framework is built upon a decomposition-reconstruction architecture inspired by multimodal unsupervised image-to-image translation.

1. Color-Content Decomposition

The model splits a distorted underwater image into two latent representations: a content code, which captures structure and texture, and a color code, which encapsulates semantic color information.

2. Explicit Latent Constraints

To ensure the color code is semantically meaningful and navigable, the authors apply a Maximum Mean Discrepancy (MMD) loss. This forces the color codes to follow a Gaussian distribution, transforming a standard restoration problem into a generative one where users can "sample" or "interpolate" through a continuous color space.

3. Functional Capabilities

As shown in Figure 1, the framework supports three primary tasks:

  • Color Enhancement (CE): A supervised path to generate a high-quality reference result.
  • Color Adaptation (CA): Users can provide a guidance image to transfer specific color characteristics to the target scene using a weight factor ($\alpha$).
  • Color Interpolation (CI): By sampling points in the latent space, the model generates a smooth spectrum of realistic color variations.
Overview of the ColorCode framework, illustrating (a) depth-dependent distortions, (b) color enhancement, (c) adaptation with guidance images, and (d) interpolation.

Key Insights: Color vs. Style

A core contribution of this work is the distinction between "color codes" and traditional "style codes." While style transfer often alters the artistic feel or texture of an image, a color code in the context of UIE must align strictly with physical or biological references to ensure accuracy.

By leveraging a "hue invariance" prior—the observation that the underlying hue of long-wavelength light is often preserved even as intensity fades—ColorCode ensures that color adaptations remain physically plausible. Figure 4 demonstrates how the proposed color code interpolation produces results that more closely match the reference compared to standard style-based methods.

Qualitative comparison of interpolation results. (a) and (b) show the difference between style and color code interpolation, while (c) compares the center of these interpolations against the reference.

Results and Performance

ColorCode was evaluated on four major benchmarks: UIEB, LSUI, EUVP, and UFO-120.

  • Quantitative Excellence: The model consistently ranks in the top three for PSNR (Peak Signal-to-Noise Ratio), SSIM (Structural Similarity), and UIQM (Underwater Image Quality Measure). On the UIEB dataset, it achieved a PSNR of 21.65, outperforming specialized architectures like UColor and WaterNet.
  • Qualitative Superiority: Visual analysis confirms that ColorCode recovers fine textures and vibrant, natural-looking colors without the artificial over-saturation common in other deep-learning methods.
  • Semantic Interpolation: The ability to "walk" through the latent space allows for smooth transitions (e.g., shifting a fish from a blue tint to its natural orange), proving the latent space is well-organized and semantically grounded.

Conclusion and Future Work

ColorCode introduces a controllable dimension to underwater image enhancement. While highly effective, the model's color adaptation feature currently relies on training data that includes at least some residual long-wavelength light to learn hue invariance. Future research will focus on extending this decomposition approach to address other complex degradations, such as non-uniform lighting and backscatter, with similar precision and user control.

Summary updated

Abstract

Underwater images often suffer from quality degradation due to absorption and scattering effects. Most existing underwater image enhancement algorithms produce a single, fixed-color image, limiting user flexibility and application. To address this limitation, we propose a method called \textit{ColorCode}, which enhances underwater images while offering a range of controllable color outputs. Our approach involves recovering an underwater image to a reference enhanced image through supervised training and decomposing it into color and content codes via self-reconstruction and cross-reconstruction. The color code is explicitly constrained to follow a Gaussian distribution, allowing for efficient sampling and interpolation during inference. ColorCode offers three key features: 1) color enhancement, producing an enhanced image with a fixed color; 2) color adaptation, enabling controllable adjustments of long-wavelength color components using guidance images; and 3) color interpolation, allowing for the smooth generation of multiple colors through continuous sampling of the color code. Quantitative and visual evaluations on popular and challenging benchmark datasets demonstrate the superiority of ColorCode over existing methods in providing diverse, controllable, and color-realistic enhancement results. The source code is available at https://github.com/Xiaofeng-life/ColorCode.
BibTeX
@misc{cong2024underwater,
  author = {Xiaofeng Cong and Jing Zhang and Yeying Jin and Junming Hou and Yu Zhao and Jie Gui and James Tin-Yau Kwok and Yuan Yan Tang},
  title = {{Underwater Organism Color Enhancement via Color Code Decomposition, Adaptation and Interpolation}},
  year = {2024},
  eprint = {2409.19685},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2409.19685},
}
Figure