Surface Normals and Shape From Water

Satoshi Murai, Meng-Yu Kuo, Ryo Kawahara, Shohei Nobuhara, Ko Nishino

Introduction

In computer vision, 3D reconstruction typically involves recovering either the 3D coordinates (depth) or the surface orientation (normals) of an object. Most existing techniques estimate one of these properties and derive the other—for instance, by integrating normals to obtain a shape. However, this dependency often leads to a loss of fine geometric detail or significant errors when the assumed surface smoothness is violated.

Underwater environments present a unique set of challenges due to light attenuation, where light is absorbed as it travels through the medium. While previous "Shape from Water" methods attempted to use this absorption to estimate depth, they were unable to independently recover surface normals. This paper introduces a novel method to simultaneously and independently recover per-pixel surface normals and depth for objects submerged in water, enabling the reconstruction of intricate, sharp features on both static and dynamic objects.

Methodology

The core of the proposed approach is the integration of the Beer-Lambert Law, which describes light attenuation in a medium, with Lambertian reflectance (shading).

Physical principles and experimental setup for shape-from-water.

Multi-Wavelength NIR and the Beer-Lambert Law

Water absorbs near-infrared (NIR) light much more aggressively than visible light, and this absorption rate varies significantly across different NIR wavelengths (e.g., 850nm vs. 950nm). By leveraging this wavelength-dependent absorption, the medium itself acts as a "depth sensor." Because NIR light dims predictably with distance, the intensity of the reflection provides a direct cue for how much water the light has traversed.

Decoupling Depth and Normals

The system utilizes at least four directional NIR light sources: one "base" light and several "auxiliary" lights with varying wavelengths or directions. The authors derive a principled theory showing that the ratio of intensities from these different light sources allows them to isolate depth. They define an "effective absorption coefficient" that accounts for the specific light path geometry.

Depth estimation is reduced to finding the root of a monotonic function, solved efficiently using numerical methods like Newton-Raphson. Once depth is known, surface normals are calculated via a simple matrix pseudo-inverse. Because depth and normals are recovered separately at every pixel, the method does not require artificial smoothness constraints, allowing it to capture sharp geometric features that are often blurred by conventional algorithms.

Experimental Validation

The authors validated their method using synthetic data, static real-world objects (such as seashells and figurines), and dynamic subjects.

Dynamic reconstruction of a swimming fish showing recovered oriented points and surface normals.

Key Results

  • Superior Accuracy: Synthetic tests demonstrated that the proposed method achieved significantly lower errors ($E_n = 1.619^\circ$) compared to deriving normals from depth ($E_n = 51.674^\circ$).
  • Noise Robustness: Increasing the number of auxiliary light sources (from 3 up to 8) was found to effectively dampen the impact of image noise on the final reconstruction.
  • High-Detail Video: A custom video-rate system, capturing six spectral bands at 14 fps, successfully reconstructed the thin, translucent fins of a swimming Siamese fighting fish. The results maintained sharp edges and creases that would typically be lost to spatial regularization.
  • Textureless Surfaces: Unlike stereo-matching or structure-from-motion, this method performs equally well on uniform, textureless surfaces as it does not rely on visual features.

Limitations and Future Work

While effective, the current model assumes surfaces are perfectly diffuse (Lambertian) and may struggle with highly specular or transparent objects. Additionally, the method assumes clear water; in turbid environments, light scattering would require a more complex radiative transfer model. Future research may extend this framework to handle non-Lambertian reflectance and explore applications in underwater archaeology and medical endoscopy.

Summary updated

Abstract

In this paper, we introduce a novel method for reconstructing surface normals and depth of dynamic objects in water. Past shape recovery methods have leveraged various visual cues for estimating shape (e.g., depth) or surface normals. Methods that estimate both compute one from the other. We show that these two geometric surface properties can be simultaneously recovered for each pixel when the object is observed underwater. Our key idea is to leverage multi-wavelength near-infrared light absorption along different underwater light paths in conjunction with surface shading. We derive a principled theory for this surface normals and shape from water method and a practical calibration method for determining its imaging parameters values. By construction, the method can be implemented as a one-shot imaging system. We prototype both an off-line and a video-rate imaging system and demonstrate the effectiveness of the method on a number of real-world static and dynamic objects. The results show that the method can recover intricate surface features that are otherwise inaccessible.
BibTeX
@inproceedings{murai2019surface,
  author = {Satoshi Murai and Meng-Yu Kuo and Ryo Kawahara and Shohei Nobuhara and Ko Nishino},
  title = {{Surface Normals and Shape From Water}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2019},
  doi = {10.1109/iccv.2019.00792},
}
Figure