Abstract
Indirect Time-of-Flight (iToF) cameras are widely used for 3D sensing due to their cost-effectiveness and high spatial resolution. However, their reliance on sequential phase captures leads to significant motion artifacts in dynamic scenes. This paper introduces iToF-flow, a novel framework that decomposes complex 3D motion into cross-mode and uni-mode flows. By leveraging a motion-insensitive Local Linear Transform (LLT) and a physics-inspired 3D CNN for photometric correction, the proposed method achieves a 38% improvement in depth accuracy and a 75% reduction in inference time, enabling high-frame-rate, artifact-free depth imaging.
Introduction: The iToF Speed Limit
The primary challenge in iToF imaging is the "speed limit" imposed by the multi-phase capture process. To compute a single depth frame, an iToF sensor typically requires four sequential measurements (phases). In dynamic environments, objects move between these captures, causing the four phases to misalign. This misalignment results in ghosting and blurring artifacts. While traditional 2D optical flow can align pixels in the XY-plane, it fails to account for intensity changes caused by motion along the Z-axis (depth-wise motion), which follows the inverse-square law of light.
Methodology: The iToF-flow Framework
To address these challenges, the authors propose the iToF-flow framework, which predicts missing phase measurements for every timestamp to boost the effective frame rate.
The framework decomposes the variation in measurements into two distinct components:
- Cross-mode Flow (LCTM): This component handles variations caused by switching measurement modes and XY-plane motion. It utilizes a Local Linear Transform (LLT) module, based on the observation that the relationship between different phase intensities in small local regions remains stable even under slight motion.
- Uni-mode Photometric Correction (UPCM): This addresses the "photometric residual"—the brightness change occurring as an object moves closer to or further from the camera. A lightweight 3D CNN is employed to estimate and correct these residuals based on physical light propagation laws.
Decoupling 3D Motion
The core innovation lies in decoupling 3D "scene flow" into manageable 2D pixel shifts and photometric intensity corrections. The LLT module provides a mathematical shortcut for alignment without requiring perfect initial pixel correspondence.
As shown in the figure above, the local linear mapping effectively captures the relationship between phases, allowing for robust alignment even near high-contrast edges where traditional methods often fail.
Experimental Results
The iToF-flow method was evaluated against state-of-the-art (SOTA) techniques like FFN and MOM using synthetic datasets (FLAT, CB-ToF, MF-ToF) and a new supplement database derived from Matterport3D LiDAR scans.
- Accuracy: The method reduced Mean Absolute Error (MAE) from ~7.58cm to 4.72cm, a 38% improvement.
- Efficiency: Computation time was slashed by 75%, from 0.32s to 0.081s per frame, moving the technology toward real-time viability.
- Robustness: Real-world tests involving fast hand gestures and moving objects demonstrated superior detail restoration and significantly fewer motion artifacts.
Conclusion
iToF-flow represents a significant step forward in high-speed 3D sensing. By integrating physical principles with deep learning, the framework overcomes the inherent temporal limitations of iToF sensors. Future work will focus on optimizing the 3D CNN components for embedded hardware and exploring applications in multi-frequency iToF systems.
Summary updated