Abstract
This paper presents a novel framework for utilizing the Directional Truncated Signed Distance Function (DTSDF) in real-time 3D mapping and tracking. By storing surface information in six cardinal directions, the DTSDF effectively resolves geometry corruption in thin objects—a common failure mode for standard TSDFs. We introduce a rendering pipeline that selectively combines directional volumes for conflict-free view generation and a Sim(3) point-to-plane ICP algorithm to handle scale discrepancies in multi-sensor registration.
Introduction
Standard 3D reconstruction techniques, such as KinectFusion, rely on Truncated Signed Distance Functions (TSDFs) to represent surfaces. While effective for large structures, TSDFs struggle with thin objects where the front and back surfaces reside within the same voxel's truncation distance. This leads to "eroded" geometry and tracking drift.
The Directional TSDF (DTSDF) was previously proposed to store these directional measurements separately. However, its adoption has been limited by the lack of efficient rendering methods for real-time tracking. This work bridges that gap, making DTSDF a viable "drop-in" replacement for standard mapping systems.
Methodology
The proposed system integrates three core technical advancements to enable robust, real-time robotic mapping:
1. DTSDF Rendering and Color Fusion
To enable tracking, the system must "render" a virtual view from the stored DTSDF. The authors developed a raycasting method that selectively combines the six directional volumes based on the camera's viewpoint. By ignoring directional data facing away from the camera (the "backside"), the system generates clean, conflict-free depth and color images.
2. Frame-to-Keyframe (f2kf) Tracking
To balance resolution and drift, the authors introduce frame-to-keyframe photometric tracking. Instead of tracking against the entire (potentially blurry) model or just the previous frame, the current RGB-D frame is compared to a high-quality keyframe, ensuring stable alignment and reducing the "blurring" effect common in frame-to-model tracking.
3. Sim(3) Point-to-Plane ICP
In multi-sensor environments, such as agricultural robots using both LiDAR and stereo cameras, calibration errors often manifest as scale differences. The authors extended standard SE(3) ICP to Sim(3), incorporating a scale parameter that allows the system to align point clouds from disparate sensors by adjusting their relative scale.
Experimental Results
The algorithm was validated on standard benchmarks (TUM RGB-D, ICL-NUIM) and a specialized agricultural dataset (PhenoRob) featuring sugar beet crops.
- Tracking Performance: DTSDF significantly improved accuracy in scenes with thin structures. In the "Zhou office" sequence, the Relative Pose Error (RPE) was reduced by nearly 50% compared to standard TSDF methods at larger voxel sizes.
- Memory Efficiency: Despite storing six directions, voxel hashing keeps memory usage within 1.5x to 2x of a standard TSDF, as most voxels only contain data from a few directions.
- Multi-Sensor Registration: The Sim(3) optimization proved essential for high-precision mapping in agricultural settings, successfully resolving scale discrepancies between different depth-sensing technologies.
Conclusion
By providing the necessary rendering math and improved tracking techniques, this work transforms the DTSDF from a theoretical storage format into a practical tool for robust robotic mapping. While the memory overhead is manageable on modern GPUs, future work will focus on further optimization for low-power mobile platforms and extending the system to handle dynamic environments.
Summary updated