TL;DR
This paper introduces a dense RGB-D SLAM system designed for indoor environments where large moving objects—like people or robots—frequently block the camera's view. Unlike traditional methods that fail when the background is obscured or require pre-trained AI to recognize objects, this approach uses a hierarchical "planes and super-pixels" strategy combined with basic robot odometry to simultaneously track multiple objects and reconstruct the static environment.
What Problem Does This Paper Address?
Most Simultaneous Localization and Mapping (SLAM) algorithms operate on a "static world" assumption: they assume that the majority of what the camera sees is stationary. When a large object moves in front of the camera, these systems often get confused, either treating the moving object as part of the map or incorrectly calculating the camera's own movement (ego-motion).
Existing solutions usually fall into two camps:
- Outlier-based: They assume moving objects are small and can be ignored as "noise." This fails when a large object (like a person standing right in front of the robot) takes up 80% of the frame.
- Semantic-based: They use deep learning (like Mask R-CNN) to identify and mask out known categories like "person" or "car." This fails if the robot encounters an object it wasn't trained to recognize.
The authors tackle the challenge of "dense" SLAM in environments where multiple large, unknown objects are moving, and the static background is no longer the dominant part of the visual input.
Methods
The researchers propose a pipeline that represents the world through a hierarchy of planes and super-pixels.
- Hierarchical Representation: Since indoor environments are full of flat surfaces (walls, floors, boxes), the system first extracts planes from the depth map. Areas that aren't flat are broken down into "super-pixels" (small, color-consistent clusters of pixels).
- Multimotion Segmentation: The system matches planes between frames and clusters them into distinct "rigid bodies" based on their movement. Instead of a binary "moving or static" choice, it treats each plane as a motion hypothesis.
- Joint Tracking and Segmentation: This is the "secret sauce." The system uses a camera motion prior (rough data from the robot's wheels or sensors) to help distinguish the camera's movement from the objects' movement. It solves an optimization problem that decides which parts of the image are static background and which are dynamic objects.
- Tracking and Reconstruction: Static parts are used to build a 3D model of the room (using a surfel-based approach), while dynamic planar objects are tracked independently as they move through space. Non-planar moving objects (like humans) are treated as outliers and removed to keep the map clean.
Datasets
The authors evaluated their method using:
- Custom Robotic Sequences (seq 1-8): Collected using an Azure Kinect DK mounted on an omnidirectional robot. They used a high-precision Vicon motion capture system to provide "ground truth" for both the camera and the moving objects (stacked boxes moved by humans or other robots).
- TUM RGB-D Dataset: Specifically the sitting_xyz and walking_xyz sequences. These are standard benchmarks used to test how SLAM handles non-planar environments and moving people.
Key Insights and Ideas
- Planes are Better than Points: In indoor SLAM, using planes as the fundamental unit is much more computationally efficient and robust than tracking thousands of individual points, especially for identifying rigid moving objects like boxes or carts.
- Odometry as a Tie-Breaker: When a large object moves, the visual data is ambiguous (is the camera moving left, or is the object moving right?). By "tightly coupling" a rough motion prior from the robot's wheels, the system can break this ambiguity even when the background is almost entirely hidden.
- Soft Clustering: Unlike previous "multimotion" methods that make hard decisions about whether two motions are the same, this paper uses a probabilistic score ($b_{ij}$) to merge planes into objects, allowing for smoother and more accurate segmentation.
Key Results
The proposed method, dubbed ours (when using the motion prior), significantly outperformed state-of-the-art methods like StaticFusion and Co-Fusion*:
- Localization Accuracy: In sequences with heavy occlusion, the system reduced the Absolute Trajectory Error (ATE) by a wide margin. For example, in "seq1," it achieved an ATE of 4.23 cm, compared to 10.4 cm for the next best baseline.
- Robustness to Drift: The system remained stable even when the camera motion prior was intentionally corrupted with significant "drift" (up to 24 cm/s), proving it doesn't need perfect sensors to work.
- Object Tracking: It successfully tracked multiple moving objects simultaneously, providing accurate trajectories for the objects themselves, not just the camera.
Limitations and Future Work
- The "Stop" Problem: If a dynamic object stops moving for a while, the system may lose track of it or struggle to distinguish it from the static background.
- Occlusion Chains: If one moving object completely hides another moving object, the system currently treats the reappearing object as a brand-new entity rather than recognizing it.
- Non-Planar Objects: While the system removes moving humans effectively, it cannot currently "track" them as distinct objects because they aren't planar. The authors plan to extend the method to track non-planar rigid bodies in the future.
Summary updated