Underwater light field imaging: A survey

Peixian Zhuang, Xinheng Zhang, Junjie Tong, Chenyu Wang, Jieyu Yuan, Yu LiCAAI 2025 · DOI

TL;DR

This paper provides the first comprehensive survey of Underwater Light Field (ULF) imaging, a technology that captures the direction of light rays rather than just their intensity. By moving from 2D pixels to 4D radiance maps, researchers are finally overcoming the "murky" challenges of the ocean—scattering, absorption, and refraction—to enable high-fidelity 3D reconstruction and long-distance underwater vision.

What Problem Does This Paper Address?

Capturing clear images underwater is notoriously difficult. Light doesn't travel in straight lines; it hits suspended particles (scattering), gets swallowed by the medium (absorption—which is why everything looks blue/green), and bends at the water-air interface (refraction).

Traditional 2D photography "flattens" the scene, integrating all incoming light angles into a single pixel. This process loses the angular information necessary to distinguish between a ray coming directly from an object and a ray that has bounced off a particle of silt. The authors address the lack of a systematic overview of how Light Field technology—which records the 4D "plenoptic function"—can be specifically adapted to the unique, high-dimensional chaos of underwater environments.

Methods

As a survey paper, the authors categorize two decades of research into a three-level framework:

  1. Foundational Theory: The transition from the 7D plenoptic function (position, angle, wavelength, time) to a manageable 4D model $H(x, y, u, v)$. It covers the evolution from early 1990s simulations to modern hardware like the Lytro Illum.
  2. Influencing Factors: A deep dive into what messes with ULF.
  3. External: How wind, waves, and algal fluorescence (which kills polarization) shift the light field.
  4. Internal: The impact of light source geometry and the inherent optical properties of the water body.
  5. Advanced Applications: The "how-to" of fixing images. This includes ray re-selection to eliminate refraction, polarization-based descattering, and using deep learning (CNNs and GANs) for image restoration and 3D reconstruction.

Datasets

While the paper is a survey rather than a benchmark study, it highlights several key data sources and hardware-specific outputs used in the field:

  • SIPPER & eHoloCam: Specialized systems for recording microscopic marine particles and plankton.
  • Lytro Illum Datasets: Many cited works utilize the Lytro light field camera, adapted for underwater use.
  • Underwater Light Field Retention Dataset: Specifically mentioned as a method/dataset for 4D light field enhancement (Ref [51]).
  • Synthetic Data: The paper notes a heavy reliance on Monte Carlo simulations and Fredholm integral equations to generate "ground truth" data where real-world underwater capture is impossible.

Key Insights and Ideas

  • The "Refraction-Free" Synthesis: One of the most elegant uses of ULF is the ability to "digitally" correct the bending of light at the water-air interface. By re-sampling geometrically accurate rays from the 4D data, researchers can synthesize images that look as if the water wasn't there at all.
  • Wind as a Variable: The paper highlights that wind speed isn't just a surface condition; it fundamentally changes how deep light can penetrate. Higher wind speeds enhance scattering, requiring entirely different calibration models for the same body of water.
  • Bio-Inspiration: A recurring theme is the move toward "bio-inspired" sensors. By mimicking the compound eyes of insects or the high-sensitivity visual structures of deep-sea fish, researchers hope to create ultra-wide-angle ULF sensors that are smaller and more efficient than current bulky camera arrays.

Key Results

The survey tracks the field's progress through several quantitative metrics (Table 3), such as UIQM (Underwater Image Quality Measure) and UCIQE (Underwater Color Image Quality Evaluation).

Key qualitative milestones include:

  • Distance Expansion: The use of "Lorentz sources" and pulse laser line scanning (PLLS) has significantly extended the effective imaging distance in turbid (cloudy) water compared to traditional cameras.
  • 3D Fidelity: ULF-based reconstruction now outperforms traditional stereo vision in "weak texture" environments (like sandy sea floors) because it can leverage angular coherence rather than just feature matching.
  • Scattering Mitigation: Modern ULF methods can isolate the "laser spot" from the scattering background, drastically improving contrast in highly turbid environments.

Limitations and Future Work

Despite the promise, the authors identify several "bottlenecks":

  • Hardware Bulk: Current ULF systems are often too large or expensive for widespread deployment on small AUVs (Autonomous Underwater Vehicles).
  • The "Data Deluge": Processing 4D data in real-time underwater requires massive computational power, which is at odds with the limited battery life of marine robots.
  • Generalization: Most models are tuned to specific water types. A "universal" ULF model that works in both the crystal-clear Caribbean and the murky Yangtze River remains elusive.

Future Directions: The authors point toward ULF Invariants—optical features that remain stable regardless of water turbulence—and the integration of Generative AI (GANs and Diffusion Models) to speed up the reconstruction of high-resolution light fields from sparse, noisy data.

Summary updated

Abstract

Underwater Light Field (ULF) imaging has emerged as a promising technology for capturing a more comprehensive array of visual information from real-world underwater environments. Unlike conventional underwater photography, which captures a two-dimensional projection of light that integrates over the angular domain, ULF imaging collects radiance from multiple directions. This enables the recovery of angular information that is typically lost in traditional imaging methods. Although ULF presents high-dimensional challenges such as inaccurate modeling and complex feature extraction, its ability to represent underwater visual data enhances the understanding of marine scenes. This, in turn, significantly improves the performance of various underwater vision tasks. The field of ULF imaging has garnered increasing attention in both the computer vision and computer graphics communities. This paper presents a comprehensive overview of the research conducted in this area over the past two decades. We focus on various aspects of ULF imaging, including ULF models, theory progression, parameter calibration, external and internal influencing factors, underwater scattering and refraction removal, underwater image enhancement/restoration, expansion of underwater imaging distance, underwater object detection, and underwater 3D reconstruction. Additionally, we analyze the current challenges facing ULF imaging technology and explore potential directions for its future development.
BibTeX
@inproceedings{zhuang2025underwater,
  author = {Peixian Zhuang and Xinheng Zhang and Junjie Tong and Chenyu Wang and Jieyu Yuan and Yu Li},
  title = {{Underwater light field imaging: A survey}},
  booktitle = {CAAI},
  year = {2025},
  doi = {10.26599/air.2025.9150051},
}
Figure