The Challenge of Dynamic Environments
Most visual SLAM (vSLAM) systems operate under a "static world" assumption, presupposing that the majority of the environment remains stationary. While outlier rejection schemes like RANSAC can handle minor movements, they often fail in highly dynamic settings such as crowded city streets or busy indoor hallways. In these scenarios, moving objects like pedestrians and vehicles can dominate the camera's field of view, leading to significant tracking drift or system failure.
Existing solutions generally fall into two categories: geometric methods and semantic methods. Geometric approaches detect motion via optical flow but are frequently noisy and struggle with precise object boundaries. Conversely, semantic methods use deep learning to mask out all potentially dynamic objects (e.g., every car in a frame). However, this often leads to "over-removal." For instance, a parked car is an excellent static landmark for SLAM; discarding it removes valuable features necessary for navigation, particularly in parking lots or residential areas.
A Multi-Stage Fusion Pipeline
To address these limitations, this research proposes a robust vSLAM system that fuses semantic information with geometric motion cues. The system introduces a three-stage preprocessing pipeline designed to integrate seamlessly with existing frameworks like ORB-SLAM2.
1. Sparse Scene Flow Clustering
The system first calculates the 3D motion (scene flow) of sparse feature points. Using Delaunay triangulation, it constructs a graph of these points and prunes edges based on motion similarity and spatial proximity. This process effectively clusters points moving together as a single rigid body.
2. Semantic Segmentation
In parallel, a Convolutional Neural Network (SegNet) provides pixel-wise labels, identifying regions belonging to "movable" categories such as vehicles and people.
3. Probabilistic Fusion
The core innovation lies in the probabilistic fusion of these two streams. Rather than using simple binary logic, the authors employ Distance Transforms and Gaussian modeling to create a probability map.
By converting binary masks into distance maps, the system creates a "soft" boundary that accounts for sensor and segmentation uncertainty. This allows the system to distinguish between "movable" objects (semantics) and "actually moving" objects (geometry). Consequently, the system can retain features on a stationary car even if it belongs to a dynamic semantic class.
Experimental Evaluation
The proposed method was evaluated across three major benchmarks: TUM RGB-D (indoor), KITTI (outdoor automotive), and ApolloScape (highly dynamic urban scenes).
- TUM Dataset: In high-dynamic "walking" sequences, the method reduced the Absolute Trajectory Error (ATE) by up to 90% compared to the baseline ORB-SLAM2.
- ApolloScape: On challenging urban sequences, translation error was reduced from 33.03% to 22.44%. Notably, semantic-only methods performed worse in some cases because they removed too many static features, validating the fusion approach.
- Ablation Studies: Results consistently showed that the combination of baseline, geometry, and semantics (B+G+S) outperformed using any single cue alone.
Key Insights and Future Directions
The research highlights that "movable" and "moving" are distinct states. A robust SLAM system must differentiate between them to maintain sufficient tracking features. By utilizing sparse scene flow and parallel processing for semantics, the system maintains near real-time efficiency.
Despite its success, the system remains dependent on the accuracy of the underlying semantic model; mislabeling static background elements as vehicles can still degrade performance. Future work may involve integrating more advanced real-time segmentation models, such as transformers, to further refine masks and improve robustness in increasingly chaotic environments.
Summary updated