Non-Rigid Shape From Water

Meng-Yu Jennifer Kuo, Ryo Kawahara, Shohei Nobuhara, Ko NishinoIEEE TPAMI 2021 · DOI · OpenReview

Abstract

We introduce a novel 3D sensing method for recovering a consistent, dense 3D shape of a dynamic, non-rigid object in water. The method reconstructs a complete (or fuller) 3D surface of the target object in a canonical frame (e.g., rest shape) as it freely deforms and moves between frames by estimating underwater 3D scene flow and using it to integrate per-frame depth estimates recovered from two near-infrared observations. The reconstructed shape is refined in the course of this global non-rigid shape recovery by leveraging both geometric and radiometric constraints. We implement our method with a single camera and a light source without the orthographic assumption on either by deriving a practical calibration method that estimates the point source position with respect to the camera. Our reconstruction method also accounts for scattering by water. We prototype a video-rate imaging system and show 3D shape reconstruction results on a number of real-world static, deformable, and dynamic objects and creatures in real-world water. The results demonstrate the effectiveness of the method in recovering complete shapes of complex, non-rigid objects in water, which opens new avenues of application for underwater 3D sensing in the sub-meter range.
BibTeX
@article{kuo2021non,
  author = {Meng-Yu Jennifer Kuo and Ryo Kawahara and Shohei Nobuhara and Ko Nishino},
  title = {{Non-Rigid Shape From Water}},
  journal = {IEEE Trans. Pattern Anal. Mach. Intell.},
  year = {2021},
  doi = {10.1109/tpami.2021.3075450},
}

Loading related content...

Figure