Underwater Light Field Retention : Neural Rendering for Underwater Imaging

Tian Ye, Sixiang Chen, Yun Liu, Erkang Chen, Yi Ye, Yuche Li

Abstract

Training robust underwater image enhancement models is hindered by the lack of paired real-world data. We present Underwater Neural Rendering (UWNR), a data-driven framework that synthesizes realistic underwater imagery from clean terrestrial images. Our approach utilizes a Natural Light Field Retention (LFR) module to capture environmental styles and a Multi-branch Hybrid Unet (MHB-Unet) for high-fidelity rendering. We introduce the Large Neural Rendering Underwater Dataset (LNRUD), featuring 50,000 diverse images, to facilitate future research. Experimental results demonstrate that UWNR outperforms existing methods in both qualitative realism and quantitative metrics.

1. Introduction

Underwater image enhancement is critical for marine exploration and robotics. However, obtaining paired "degraded-clean" images in situ is nearly impossible. While synthetic data offers a solution, traditional methods based on simplified physical models or unstable Generative Adversarial Networks (GANs) often fail to capture the complex, non-uniform nature of underwater light scattering. This work proposes UWNR to bridge the gap between synthetic and real-world underwater imagery through implicit neural rendering.

2. Methodology

The UWNR framework avoids explicit physical modeling, instead learning degradation patterns from authentic underwater exemplars.

2.1 Natural Light Field Retention (LFR)

Inspired by Retinex theory, the LFR module extracts the ambient "style" of a real underwater scene. By applying multi-scale Gaussian low-pass filters, the module generates a light field map that preserves color casts and haze while removing high-frequency object details. This map serves as a spatial template for the rendering process.

2.2 Multi-branch Hybrid Unet (MHB-Unet)

The rendering engine, MHB-Unet, integrates three inputs: a clean terrestrial image, an estimated depth map (simulating distance-dependent attenuation), and the LFR map. The architecture employs spatial and channel attention mechanisms to prioritize regions with significant degradation and critical color channels. The model is optimized using a composite loss function including $L_1$ reconstruction, perceptual loss, Underwater Dark Channel (UDCP) loss, and a novel Light Field Consistency loss.

3. The LNRUD Dataset

To support large-scale training, we developed the Large Neural Rendering Underwater Dataset (LNRUD). By applying 5,000 authentic underwater light fields to clean images from the NYU and RESIDE datasets, we generated 50,000 high-quality synthetic pairs. LNRUD is significantly larger and more diverse than existing benchmarks, providing a superior training ground for enhancement algorithms.

4. Experimental Results

UWNR demonstrates significant improvements over state-of-the-art methods like UWGAN and UISA.

Qualitative comparison between UWGAN, UISA, and the proposed UWNR method across indoor and outdoor scenes.

4.1 Quantitative Performance

On the NYU dataset, UWNR achieved a Fréchet Inception Distance (FID) of 221.93, outperforming UWGAN (236.23). This indicates that the distribution of UWNR-generated images is much closer to real underwater photography.

4.2 Qualitative Realism

As shown in the visual comparisons, UWNR avoids the artificial "over-greening" typical of GANs and the unrealistic brightness of physical models. The resulting images exhibit natural haze gradients and color attenuation consistent with real-world physics.

4.3 Computational Efficiency

The model is highly optimized, containing only 11.57M parameters. It achieves an inference time of 0.0023 seconds for a 1024x1024 image, enabling rapid generation of massive datasets.

5. Conclusion

UWNR provides a robust, efficient, and realistic solution for underwater image synthesis. By moving from explicit physical equations to implicit neural rendering with Light Field Retention, we provide a more accurate representation of underwater environments. The LNRUD dataset stands as a valuable resource for the community to advance underwater vision tasks.

Summary updated

Abstract

Underwater Image Rendering aims to generate a true-tolife underwater image from a given clean one, which could be applied to various practical applications such as underwater image enhancement, camera filter, and virtual gaming. We explore two less-touched but challenging problems in underwater image rendering, namely, i) how to render diverse underwater scenes by a single neural network? ii) how to adaptively learn the underwater light fields from natural exemplars, i,e., realistic underwater images? To this end, we propose a neural rendering method for underwater imaging, dubbed UWNR (Underwater Neural Rendering). Specifically, UWNR is a data-driven neural network that implicitly learns the natural degenerated model from authentic underwater images, avoiding introducing erroneous biases by hand-craft imaging models. Compared with existing underwater image generation methods, UWNR utilizes the natural light field to simulate the main characteristics ofthe underwater scene. Thus, it is able to synthesize a wide variety ofunderwater images from one clean image with various realistic underwater images. Extensive experiments demonstrate that our approach achieves better visual effects and quantitative metrics over previous methods. Moreover, we adopt UWNR to build an open Large Neural Rendering Underwater Dataset containing various types of water quality, dubbed LNRUD. The source code and LNRUD are available at https: //github.com/Ephemeral182/UWNR.
BibTeX
@inproceedings{ye2022underwater,
  author = {Tian Ye and Sixiang Chen and Yun Liu and Erkang Chen and Yi Ye and Yuche Li},
  title = {{Underwater Light Field Retention : Neural Rendering for Underwater Imaging}},
  booktitle = {Workshops @ CVPR},
  year = {2022},
  doi = {10.1109/cvprw56347.2022.00064},
}
Figure