Real-Time Dynamic 3D Shape Reconstruction with SWIR InGaAs Camera

Cheng Fei, Yanyang Ma, Shan Jiang, Junliang Liu, Baoqing Sun, Yongfu Li, Yi Gu, Xian Zhao, Jiaxiong Fang2020 · DOI

TL;DR

This paper presents a real-time dynamic 3D shape reconstruction system using a Short-Wave Infrared (SWIR) InGaAs camera and an improved Fourier-Transform Profilometry (FTP) method. The system achieves high accuracy and speed, making it suitable for low-illumination monitoring applications where visible light is not ideal.

What Problem Does This Paper Address?

Accurately reconstructing the 3D shape of objects in real-time, especially in challenging environments like low illumination, is a significant challenge. Traditional optical 3D shape reconstruction methods often rely on visible light, which can be problematic for applications requiring stealth or where visible light sources are impractical. Furthermore, capturing dynamic scenes in 3D requires high-speed acquisition and processing. This paper aims to bridge this gap by developing a system that can perform real-time dynamic 3D shape reconstruction using SWIR light, which offers advantages like eye-safety and better concealment, making it ideal for monitoring in low-light conditions.

Methods

The core of the system is an improved Fourier-Transform Profilometry (FTP) method. FTP is a single-shot 3D reconstruction technique, meaning it can extract depth information from a single fringe pattern image, which is crucial for high-speed dynamic reconstruction. The authors' improvement focuses on:

  • Extracting the 2D background fringe pattern from the deformed fringe pattern image without needing a separate background image capture. This significantly speeds up the process.
  • Reducing phase unwrapping errors by leveraging the carrier frequency, which is beneficial for obtaining more accurate depth maps.

The system utilizes a crossed-optical-axes geometry for the projector and camera. A 1550-nm SWIR fiber laser projects fringe patterns onto the target, and a custom-developed SWIR InGaAs camera captures the deformed fringe patterns. The captured images are then processed using the improved FTP algorithm to reconstruct the 3D shape.

Datasets

The paper does not explicitly mention or introduce new benchmark datasets. Instead, it uses:

  • Plaster models of an eye, doll, hand, and lip.
  • A real human face.
  • A plastic model shaped like a gourd for static accuracy evaluation.
  • A pendulum ball for real-time dynamic accuracy evaluation.

These are used as targets for demonstrating the system's capabilities and evaluating its accuracy under various conditions.

Key Insights and Ideas

  • SWIR for Stealth and Safety: The use of SWIR light is highlighted as a key advantage. The projected fringe patterns are invisible to the human eye in visible light images, making the system suitable for covert monitoring. Additionally, SWIR light is eye-safe, which is important for applications involving human subjects.
  • Real-time Dynamic Reconstruction is Achievable: The system demonstrates the ability to reconstruct 3D shapes of moving objects at a frame rate of 25 Hz, opening doors for real-time monitoring of dynamic scenes.
  • Improved FTP Efficiency: The modifications to the FTP method significantly reduce processing time by eliminating the need for a separate background image capture, a critical factor for real-time performance.
  • Accuracy in Low Reflectance Areas: While the system achieves good accuracy, it acknowledges that areas with low reflectance (like the edges of the gourd model) can lead to larger depth errors. However, in areas with uniform reflectance and no abrupt depth changes, the accuracy is very high.

Key Results

  • Static 3D Shape Reconstruction:
    • Maximum depth error for a plastic model (36 mm depth) is 1.15 mm.
  • The system successfully reconstructed complex shapes like a human face, with details like eyes, nose, and mouth being well-captured.
  • Real-time Dynamic 3D Shape Reconstruction:
    • Achieved a frame rate of 25 Hz.
  • For a hemisphere (35 mm depth), the maximum depth error among all frames (excluding edge areas) is 1.42 mm.
  • The maximum error of the average of all frames in depth is 0.52 mm.
  • The system demonstrated robust reconstruction of a swinging pendulum ball.

Limitations and Future Work

  • Edge Errors: The paper acknowledges that areas with abrupt depth changes and low reflectance (e.g., edges of objects) can lead to significant depth errors. This is partly due to the intrinsic limitations of FTP and the nature of light reflection.
  • Reflectance Dependence: Accuracy is shown to be influenced by the reflectance of the target surface. Areas with uniform, high reflectance yield better results.
  • Future Work: The authors suggest that the system's capabilities, particularly its real-time dynamic 3D shape reconstruction in SWIR, can be further pushed into broader monitoring applications due to the unique characteristics of SWIR. Further improvements in handling low-reflectance areas and complex geometries could enhance its applicability.

Summary updated

BibTeX
@misc{fei2020real,
  author = {Cheng Fei and Yanyang Ma and Shan Jiang and Junliang Liu and Baoqing Sun and Yongfu Li and Yi Gu and Xian Zhao and Jiaxiong Fang},
  title = {{Real-Time Dynamic 3D Shape Reconstruction with SWIR InGaAs Camera}},
  year = {2020},
  doi = {10.3390/s20020521},
}
Figure