Learning to Complete Object Shapes for Object-level Mapping in Dynamic Scenes

Binbin Xu, Andrew J. Davison, Stefan Leutenegger

Introduction

Simultaneous Localization and Mapping (SLAM) is a cornerstone of robotic autonomy. However, most conventional SLAM systems are predicated on the "static world" assumption. In dynamic environments—where people move or objects are displaced—traditional algorithms often struggle, treating moving pixels as outliers to be discarded. This research presents a novel object-level SLAM system that moves beyond simple tracking. By integrating category-level shape priors with real-time depth measurements, the system "completes" the hidden geometry of dynamic objects, creating watertight 3D models that enhance both reconstruction quality and tracking robustness.

The Problem: Partial Observations and Hollow Shells

Existing dynamic SLAM systems typically map only the visible surfaces of objects. This results in "hollow shells" or sparse point clouds. When an object is partially occluded—for instance, a chair partially hidden by a person—these systems often lose track or produce fragmented maps. The authors posit that a robot's understanding of an object should include its complete geometry. Knowing that a chair has four legs, even when only two are visible, provides a more stable geometric anchor for tracking and interaction.

Methodology: Dual-Node Representation

The proposed system utilizes a dual-node architecture to represent object geometry, balancing general category knowledge with specific instance details.

1. The Prior Node

The system employs a DeepSDF network to generate an initial "best guess" of the object's shape based on its detected category (e.g., chair). This prior serves as a template, providing a rough initialization for the object's pose and geometry.

2. The Posterior Node

To capture the unique characteristics of a specific object, a refinement step is introduced. The system processes Truncated Signed Distance Function (TSDF) data through a 3D-UNet and a completion network (CONet). Unlike methods that merely interpolate between training shapes, this approach conditions the completion on the actual fused volume, allowing the system to reconstruct fine-grained details not present in the initial prior.

The SLAM Pipeline

The workflow begins with Segmentation using Mask R-CNN, followed by Data Association to link detections to the existing map. A key innovation is the Joint Optimization phase, where the system simultaneously optimizes the object’s pose and its latent geometry using geometric residuals and differentiable rendering. Finally, a Two-Stage Refinement process ensures that the coarse estimation from the shape prior is updated with dense, fused depth measurements to fill in occluded regions.

Experimental Evaluation

The researchers evaluated their system using ShapeNet (specifically the "chair" category) for training, alongside a custom Synthetic Moving Objects Dataset created in Blender to test various motion speeds. Qualitative tests were also conducted on real-world RGB-D sequences involving human-object interactions.

Key Results

  • Reconstruction Accuracy: The method significantly outperformed traditional TSDF-Fusion and DeepSDF in limited-view scenarios (e.g., 1 or 5 frames). It achieved superior Intersection over Union (IoU) and lower Chamfer Distance, indicating a high degree of fidelity to the ground truth.
  • Tracking Performance: The system maintained an Absolute Trajectory Error (ATE) of approximately 0.03m. Notably, it remained robust even as the temporal gap between keyframes increased, a scenario where standard ICP-based methods typically fail.
  • Occlusion Handling: By leveraging instance-level understanding, the system can effectively ignore "interference" from humans, maintaining a clean and complete reconstruction of the object being tracked.

Discussion and Limitations

While the system produces high-quality, watertight models, it currently faces computational challenges. The Python-based prototype runs at approximately 1.3 seconds per frame on an RTX 3090, which is below the threshold for real-time robotics applications. Furthermore, the system is currently category-dependent, requiring pre-trained priors for each object class.

Conclusion

This research demonstrates that shape completion is a powerful tool for dynamic SLAM. By imagining the unseen, robots can achieve more stable tracking and more useful environmental representations. Future work will focus on optimizing the implementation for real-time performance and extending the framework into a full graph-based SLAM system with global consistency and loop closure capabilities.

Summary updated

BibTeX
@inproceedings{xu2022learning,
  author = {Binbin Xu and Andrew J. Davison and Stefan Leutenegger},
  title = {{Learning to Complete Object Shapes for Object-level Mapping in Dynamic Scenes}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2022},
  doi = {10.1109/iros47612.2022.9981545},
}
Figure