Pulse Based Time-of-Flight Range Sensing

Hamed Sarbolandi, Markus Plack, Andreas Kolb

TL;DR

This paper provides a comprehensive evaluation of a Pulse-Based Time-of-Flight (PB-ToF) camera prototype, highlighting its strengths and weaknesses compared to the more common Amplitude Modulated Continuous-Wave (AMCW-ToF) approach. The authors introduce novel methods for optimizing PB-ToF camera operation and simulating its behavior, paving the way for improved depth sensing applications.

What Problem Does This Paper Address?

Depth sensing is crucial for many computer vision applications, from robotics and augmented reality to human-computer interaction. While Amplitude Modulated Continuous-Wave Time-of-Flight (AMCW-ToF) cameras (like those found in the Kinect) are widely used, they suffer from inherent limitations such as multi-path effects and systematic distance errors. Pulse-Based Time-of-Flight (PB-ToF) offers an alternative approach with potentially different characteristics, but has seen less in-depth evaluation. This paper aims to fill that gap by thoroughly analyzing a specific PB-ToF camera prototype, identifying its error sources, and proposing methods to enhance its performance.

Methods

The researchers conducted a detailed evaluation of a Hamamatsu S11963-01CR PB-ToF camera prototype. They systematically investigated several error sources common to ToF systems, including:

  • Temperature Drift: Monitoring distance measurements over time to assess system stability during warm-up.
  • Systematic Distance Error & Planarity: Using a linear rail to track distances to a planar wall and analyze deviations and out-of-plane errors.
  • Depth Inhomogeneity & Multi-Path Effects: Employing a reflective board rotated at various angles against a scattering background to simulate complex light paths.
  • Intensity-Related Error: Assessing how reflectivity affects range measurements.
  • Motion Artefacts: Using a rotating Siemens star to evaluate performance in dynamic scenes.

Beyond evaluation, the paper introduces novel concepts:

  • Optimized Operation Range: Analyzing gate values as a function of distance and intensity to identify regions with less standard deviation and virtually shifting the camera's operation range for improved performance.
  • PB-ToF Sensor Simulation: Developing a simulator based on measured gate response and illumination signal shapes to reproduce observed depth deviations and validate the model.

Datasets

The paper does not explicitly mention using pre-existing datasets for training. Instead, it relies on data collected from controlled experimental setups designed to evaluate specific error sources. These setups include:

  • Device Warm-Up: A planar wall observed under controlled temperature conditions.
  • Rail Depth Tracking: A planar wall observed at various distances using a motorized linear rail.
  • Reflective Board: A rotating reflective board against a scattering background.
  • Turning Siemens Star: A rotating Siemens star to simulate dynamic scenes.
  • Intensity Checkerboard: A checkerboard with varying gray levels.

The data collected from these scenarios serve as benchmarks for evaluating the camera's performance and validating the proposed simulation model.

Key Insights and Ideas

  • PB-ToF vs. AMCW-ToF Trade-offs: While the PB-ToF prototype exhibits higher noise levels than commercial AMCW-ToF cameras like the KinectToF, it shows promising results in certain areas, suggesting a different set of strengths and weaknesses.
  • Importance of Gate Response Characterization: Accurately measuring and modeling the camera's gate response functions is crucial for understanding and simulating its behavior, particularly for predicting depth deviations.
  • Virtual Operation Range Shifting: The ability to virtually shift the camera's operation range based on pre-knowledge of expected distances and intensities offers a practical way to improve standard deviation in specific scenarios.
  • Simulation as a Validation Tool: The developed PB-ToF simulator, based on measured signal shapes, successfully reproduces observed depth deviations, providing a valuable tool for understanding error sources and testing optimization strategies.
  • Intensity-Related Error Less Pronounced: Unlike some earlier AMCW-ToF devices, this PB-ToF prototype shows less prominent intensity-related errors, which is a positive sign.

Key Results

  • Temperature Drift: The PB-ToF prototype shows a higher level of noise (SD) compared to the KinectToF, though its temporal stability is qualitatively assessed.
  • Distance Error: After per-pixel correction, the distance error in the image center is generally below 1 cm up to 4 m.
  • Planarity Error: The planarity error remains relatively constant at around 2 cm over a 3.5 m distance.
  • Motion Artefacts: Significant loss of foreground pixels is observed at jumping edges for higher speeds, but background pixels are detected with higher robustness.
  • Multi-Path Interference: The multi-path interference of the PB-ToF prototype is comparable to the KinectToF in terms of mean range error.
  • Simulation Accuracy: The simulator, using measured gate response and illumination signals, reproduces measured depth deviations with good quality.
  • Operation Range Optimization: Shifting the operation range can lead to a moderate improvement in standard deviation, though it requires prior knowledge of the expected range and intensity.

Limitations and Future Work

  • High Noise Level: The authors acknowledge that the prototype camera has a significantly higher noise level (SD) compared to commercial products like the KinectToF.
  • Limited Illumination Power: The Hamamatsu camera's illumination is considerably less powerful than the KinectToF's, which is noted as a qualitative difference.
  • Prototype Nature: The observed deviations from ideal behavior (e.g., non-symmetric gate responses) are attributed to the prototype nature of the camera, suggesting that optimized commercial versions might perform better.
  • Future Work: The paper suggests that the developed simulation approach and the optimization strategies for the operation range are applicable to AMCW-ToF cameras as well. Further research could focus on developing more advanced calibration techniques and exploring the full potential of PB-ToF cameras for various applications. The authors also note that a direct quantitative comparison with the KinectToF is challenging due to differences in hardware, operation modes, and potential differences in how invalid pixels are classified.

Summary updated

Abstract

Pulse-based Time-of-Flight (PB-ToF) cameras are an attractive alternative range imaging approach, compared to the widely commercialized Amplitude Modulated Continuous-Wave Time-of-Flight (AMCW-ToF) approach. This paper presents an in-depth evaluation of a PB-ToF camera prototype based on the Hamamatsu area sensor S11963-01CR. We evaluate different ToF-related effects, i.e., temperature drift, systematic error, depth inhomogeneity, multi-path effects, and motion artefacts. Furthermore, we evaluate the systematic error of the system in more detail, and introduce novel concepts to improve the quality of range measurements by modifying the mode of operation of the PB-ToF camera. Finally, we describe the means of measuring the gate response of the PB-ToF sensor and using this information for PB-ToF sensor simulation.
BibTeX
@inproceedings{sarbolandi2018pulse,
  author = {Hamed Sarbolandi and Markus Plack and Andreas Kolb},
  title = {{Pulse Based Time-of-Flight Range Sensing}},
  booktitle = {Sensors},
  year = {2018},
  url = {https://openreview.net/forum?id=GbwCivYlZWKS},
}
Figure