Polarimetric iToF: Measuring High-Fidelity Depth Through Scattering Media

Daniel S. Jeon, Andreas Meuleman, Seung-Hwan Baek, Min H. KimCVPR 2023 · DOI

TL;DR

This paper introduces a novel polarimetric indirect Time-of-Flight (iToF) imaging method that significantly improves depth accuracy in scattering media like fog. By leveraging polarization properties and a new scattering-aware computational model, it overcomes the limitations of conventional iToF systems, enabling robust 3D perception in challenging environments.

What Problem Does This Paper Address?

Indirect Time-of-Flight (iToF) imaging is a cost-effective technology for capturing dense depth information, widely used in applications like autonomous driving and robotics. However, a major challenge for iToF is "multipath interference" (MPI), where light bounces multiple times before returning to the sensor, leading to inaccurate depth measurements. This problem is severely exacerbated in the presence of scattering media, such as fog, where light scatters extensively, making it difficult to distinguish direct reflections from scattered light. Conventional iToF cameras struggle to provide reliable depth data in such conditions, limiting their deployment in real-world scenarios like driving in fog or underwater navigation.

Methods

The core innovation of this work lies in a new polarimetric iToF imaging method that combines optical polarization filtering with a novel computational model to mitigate MPI in scattering media.

Here's a breakdown of their approach:

  • Polarization Filtering: The setup utilizes linear polarizers on both the illumination source and the detector. By configuring these polarizers in a "cross-polarization" setup (perpendicular orientation), they aim to filter out backscattered light that tends to retain the original polarization state of the illumination. This is a standard technique, but the authors note its limitations.
  • Scattering-Aware Computational Model: The key novelty is a computational model that accounts for the depolarization of light as it scatters. They observe that while cross-polarization filters out some scattered light, a significant portion of backscattered light becomes unpolarized due to multiple scattering events. Their model specifically addresses this unpolarized backscattered light.
  • Phasor Representation: They adopt a phasor representation for iToF signals, which captures amplitude, phase, and offset. This allows them to model the distortion introduced by scattering.
  • Estimating Unpolarized Backscatter:
    • They first estimate the polarized backscattered light by subtracting the cross-polarized measurement from a parallel-polarized measurement (Polarization-Difference Imaging - PDI).
  • Crucially, they then develop a model to estimate the unpolarized backscattered light. This involves modeling the scattering decay (how intensity and degree of polarization decrease with distance) and then estimating the phase and amplitude of this unpolarized component.
  • Scattering Removal: Once the unpolarized backscattered light's phasor is estimated, it's computationally subtracted from the initial cross-polarized measurements. This effectively removes the scattering-induced distortion, yielding a more accurate depth estimate.

Datasets

The paper does not explicitly mention training datasets. Instead, it focuses on experimental validation using a customized off-the-shelf iToF camera and scenes with varying fog densities. They generated ground-truth depth maps for evaluation.

Key Insights and Ideas

  • Unpolarized Backscatter is the Key: The most significant insight is that simply using cross-polarization isn't enough because multiple scattering depolarizes light. The real challenge and the paper's focus is on accurately modeling and removing this unpolarized scattered light.
  • Polarization and Scattering Theory Synergy: The method effectively bridges optical polarization principles with scattering theory to create a robust computational model.
  • Phasor Model for Distortion: Using the phasor representation provides a powerful framework to mathematically describe and then invert the depth distortion caused by scattering.
  • Robustness Across Fog Densities: The method demonstrates consistent performance across a range of fog densities, highlighting its practical applicability.

Key Results

  • Significant Depth Accuracy Improvement: The proposed polarimetric iToF method achieves substantially higher depth accuracy compared to conventional iToF methods (both parallel and cross-polarized) in the presence of fog.
  • Quantitative Performance: In Table 1, the Root Mean Square Error (RMSE) for their method is significantly lower than conventional iToF across thin, medium, and thick fog densities (e.g., 1.65 cm vs. 5.46 cm for medium fog).
  • Qualitative Results: Figures 6, 7, 10, 12-21 visually demonstrate the superior depth reconstruction quality of their method, especially in dense fog where conventional methods fail completely. Their depth maps are much closer to the ground truth.
  • Ambient Light Handling: The method also accounts for ambient light, further improving accuracy (RMSE of 1.88 cm without ambient light and 2.08 cm with ambient light, compared to 11.62 cm for conventional ToF).

Limitations and Future Work

  • Fog Homogeneity Assumption: The method assumes the participating medium (fog) is homogeneous. In reality, fog can be heterogeneous, which might affect performance.
  • 2π Ambiguity: Like other indirect ToF systems, it faces the 2π phase ambiguity, limiting the maximum measurable depth. While they argue this is often not the critical limitation in practice, it's an inherent constraint.
  • Spatial Regularization: The current method estimates depth per-pixel. The authors suggest that incorporating spatial regularization or smoothing techniques (like guided filters) could further improve depth denoising.
  • Random Polarization: The model might not be applicable to deterministic scattering of polarized light, which can occur in specific scenarios (e.g., near grazing angles with skylight).
  • Future Work: The authors suggest leveraging micro-polarizer pixel arrays on iToF imaging for dynamic scene capture and developing efficient denoising algorithms for reduced photon counts. They also mention extending the method to handle subsurface scattering and sparse k-bounce interference.

Summary updated

BibTeX
@inproceedings{jeon2023polarimetric,
  author = {Daniel S. Jeon and Andreas Meuleman and Seung-Hwan Baek and Min H. Kim},
  title = {{Polarimetric iToF: Measuring High-Fidelity Depth Through Scattering Media}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2023},
  doi = {10.1109/cvpr52729.2023.01189},
}
Figure