Quantified, Interactive Simulation of AMCW ToF Camera Including Multipath Effects

David Bulczak, Martin Lambers, Andreas Kolb

TL;DR

This paper introduces a novel, interactive, GPU-accelerated simulator for AMCW Time-of-Flight (ToF) cameras that accurately models multipath interference and sensor noise. Crucially, it presents the first quantified comparison of ToF simulators by incorporating real-world near-infrared BRDF measurements of purchasable materials, enabling more realistic scene creation and validation.

What Problem Does This Paper Address?

Time-of-Flight (ToF) cameras are increasingly popular for depth sensing in applications like robotics and automotive systems. However, their accuracy is hampered by several error sources, most notably multipath interference (MPI) – where light bounces off multiple surfaces before reaching the sensor – and motion artifacts. Developing better ToF cameras and algorithms requires robust simulation tools that can accurately reproduce these effects. Existing simulators often struggle with computational cost, lack realistic material properties, or don't provide quantitative comparisons to real-world data. This paper aims to bridge this gap by providing an interactive, physically-based simulator that includes MPI and is validated against real measurements using accurate material properties.

Methods

The core of the simulation is an extension of the physically-based model by Lambers et al. [2]. Key innovations include:

  • Interactive Multipath Simulation: They adapt the Reflective Shadow Map (RSM) algorithm [6] for GPU-based, single-bounce indirect illumination simulation. This approach is significantly faster than global illumination techniques like bidirectional path tracing, enabling interactive frame rates.
  • Physically-Based Sensor Model: The simulation models light propagation down to the charge level accumulated in individual sensor pixels, accounting for quantum efficiency and noise.
  • Realistic Material Properties: A significant contribution is the measurement and incorporation of Bidirectional Reflectance Distribution Functions (BRDFs) for purchasable materials in the near-infrared (NIR) spectrum (850 nm), which is crucial for accurate simulation of light interaction with surfaces.
  • Noise Modeling: The simulator includes a realistic noise model that accounts for electronic and optical shot noise, which is particularly important in low-light conditions.

Datasets

  • BRDF Measurements: The authors introduce a new dataset of isotropic BRDF measurements at 850 nm for several standard, purchasable materials (e.g., PLEXIGLAS®, PVC rigid foam). This dataset is made publicly available.
  • Real-World Camera Data: Depth images captured by a PMD CamCube 3.0 ToF camera were used for quantitative evaluation and comparison with the simulator's output.
  • Synthetic Scenes: The authors created synthetic scenes using the measured BRDFs and the simulator to match real-world test scenes.

Key Insights and Ideas

  • The Importance of NIR BRDFs: Simulating ToF cameras accurately requires material properties at the camera's operating wavelength (NIR). Generic BRDFs or Lambertian models are insufficient for quantitative comparisons.
  • Single-Bounce MPI is Sufficient for Many Scenarios: While higher-order bounces exist, the paper demonstrates that simulating single-bounce indirect illumination significantly improves accuracy compared to direct-only simulation, especially in corner scenes where MPI is pronounced.
  • Interactive Simulation is Achievable: By leveraging image-space techniques like RSM, it's possible to achieve interactive frame rates for ToF simulation, a significant improvement over hours-long computations for global illumination methods.
  • Quantified Comparison is Key: The paper emphasizes the need for quantitative comparisons between simulated and real data, moving beyond qualitative visual assessments to truly validate simulation models.

Key Results

  • The proposed simulator, SimSingle (including single-bounce MPI), significantly outperforms SimDirect (direct illumination only) in terms of accuracy, reducing Mean Absolute Error (MAE) by over 50% in corner scenes.
  • The simulator achieves interactive frame rates (around 80 ms per frame for accumulation after RSM computation), whereas previous MPI simulation methods required hours per image.
  • Quantitative evaluations show that the simulated depth data closely matches real CamCube measurements when using the BRDF-based materials and the single-bounce MPI model.
  • The noise model in SimSingle produces comparable noise levels to real ToF measurements, whereas SimDirect shows a higher relative impact of noise due to lower charge accumulation.

Limitations and Future Work

  • Single-Bounce Limitation: The simulation currently only accounts for single-bounce indirect illumination. The authors acknowledge that in highly reflective scenarios (e.g., with aluminum cubes), this limitation becomes apparent, and higher-order multipath effects are not captured.
  • Other Sensor Types: The authors suggest adapting the simulation technique to other ToF sensor types, such as pulse-based ToF cameras, which are also affected by multipath effects.
  • Wavelength Extension: The BRDF database could be extended to cover other wavelengths relevant to different ToF sensors.

Summary updated

Abstract

In the last decade, Time-of-Flight (ToF) range cameras have gained increasing popularity in robotics, automotive industry, and home entertainment. Despite technological developments, ToF cameras still suffer from error sources such as multipath interference or motion artifacts. Thus, simulation of ToF cameras, including these artifacts, is important to improve camera and algorithm development. This paper presents a physically-based, interactive simulation technique for amplitude modulated continuous wave (AMCW) ToF cameras, which, among other error sources, includes single bounce indirect multipath interference based on an enhanced image-space approach. The simulation accounts for physical units down to the charge level accumulated in sensor pixels. Furthermore, we present the first quantified comparison for ToF camera simulators. We present bidirectional reference distribution function (BRDF) measurements for selected, purchasable materials in the near-infrared (NIR) range, craft real and synthetic scenes out of these materials and quantitatively compare the range sensor data.
BibTeX
@inproceedings{bulczak2018quantified,
  author = {David Bulczak and Martin Lambers and Andreas Kolb},
  title = {{Quantified, Interactive Simulation of AMCW ToF Camera Including Multipath Effects}},
  booktitle = {Sensors},
  year = {2018},
  url = {https://openreview.net/forum?id=XmxaAUgjojnm},
}
Figure