Traditional Simultaneous Localization and Mapping (SLAM) and visual localization algorithms operate under a comforting fiction: the world is completely static. When a camera moves through an environment, tracking algorithms assume that any change in a feature's pixel coordinates is purely a function of the camera's own ego-motion.
If an object moves independently—such as a walking pedestrian or a passing car—the geometric constraints underpinning bundle adjustment or Extended Kalman Filtering (EKF) break down. The optimization engine attempts to minimize reprojection errors by distorting the estimated camera trajectory, leading to tracking drift or catastrophic localization failure. To build systems that work in the real world, we must find ways to isolate the static background from the dynamic foreground.
Hard Filtering via Semantic and Geometric Fusion
The most direct way to handle dynamic environments is to identify and discard features belonging to moving objects before they reach the tracking pipeline. This can be achieved by leveraging semantic priors. If a system knows what a human or a vehicle looks like, it can preemptively ignore features detected on them.
For example, Jia Liu 0034 et al. demonstrate this in VSLAM method based on object detection in dynamic environments by combining YOLOv3 object detection with a Gaussian Mixture Model (GMM) background representation to isolate and filter out dynamic objects. Similarly, Qi Zhang 0072 et al. leverage deep learning object detection alongside structural point, line, and plane features in SLAM2: Simultaneous Localization and Multimode Mapping for indoor dynamic environments to achieve highly accurate indoor tracking.
However, a purely semantic approach is often wasteful. A parked car or a chair is semantically "movable" but geometrically static, providing valuable localization anchors. To avoid discarding useful information, modern frameworks fuse semantics with geometric verification.
[Input Frame] ---> [Semantic Segmentation] ---> [Potential Dynamic Masks]
|
v
[Optical Flow / Geometry Verification] --------> [Actual Moving Objects Filtered]
Junhao Cheng et al. implement this paradigm in DM-SLAM: A Feature-Based SLAM System for Rigid Dynamic Scenes by combining instance segmentation with optical flow, ensuring that only objects exhibiting actual independent motion are masked out. This multi-modal approach is echoed by Jian Liu et al. in Towards Biologically-Inspired Visual SLAM in Dynamic Environments: IPL-SLAM with Instance Segmentation and Point-Line Feature Fusion, who pair instance segmentation with adaptive point-line feature fusion, and Chengqun Song et al. in PSMD-SLAM: Panoptic Segmentation-Aided Multi-Sensor Fusion Simultaneous Localization and Mapping in Dynamic Scenes, who utilize panoptic segmentation alongside multi-sensor fusion (LiDAR, camera, IMU) to isolate dynamic object trajectories.
What happens if semantic labels are unavailable, or an object does not fit a predefined category? The authors of RGB-D SLAM in Dynamic Environments Using Point Correlations (Weichen Dai et al.) solve this by analyzing point correlations within a sparse graph. Because static points maintain rigid geometric relationships relative to one another across frames while dynamic points violate these correlations, the system can segment out moving elements purely through graph topology. In a similar vein, Bo Wang et al. propose a bidirectional scoring strategy in A Bidirectional Scoring Strategy-Based Transformation Matrix Estimation of Dynamic Factors in Environmental Sensing to filter out dynamic factors during transformation matrix estimation.
Soft Weighting and Temporal Reliability
Binary masking can introduce instability if an object is misclassified or partially occluded. A more robust alternative is soft weighting, where features are down-weighted based on their likelihood of being dynamic.
Early work by Seok‐Han Lee et al. in Stable Stereo based EKF-SLAM in Dynamic Situation introduced a "confidence map" derived from stereo vision to down-weight depth information from dynamic regions. Modern sensors offer new ways to compute these weights; for instance, Jiafeng Huang et al. develop E2-VINS: An Event-Enhanced Visual–Inertial SLAM Scheme for Dynamic Environments, which uses the asynchronous stream of event cameras to assign "dynamicity weights" to visual features, filtering out moving objects in dense traffic or night driving.
Dynamics also manifest over longer timescales—such as a retail store where items are moved daily or weekly. Alexandros Rotsidis et al. address this in ExMaps: Long-Term Localization in Dynamic Scenes using Exponential Decay by modeling the reliability of feature points using an exponential decay function, maintaining a "live map" that naturally forgets transient features over time. On a frame-by-frame level, Haolan Zhang et al. introduce SR-SLAM: Scene reliability-based RGB-D SLAM in diverse environments, which computes an overall scene reliability assessment to adaptively guide feature selection and pose refinement strategies based on environmental conditions.
Radiance Fields and Next-Generation Representations
The shift from sparse point clouds to dense neural scene representations like 3D Gaussian Splatting (3DGS) has opened a new frontier for dynamic SLAM. Instead of filtering points before mapping, these systems model the environment as a collection of probabilistic Gaussians, allowing the optimization process itself to resolve dynamics.
[Multi-View Images] ---> [3D Gaussian Splatting] ---> [Bayesian Filtering] ---> [Static Scene Model]
|
v
[Prune Dynamic Artifacts]
Shuangfeng Wei et al. introduce BDGS-SLAM: A Probabilistic 3D Gaussian Splatting Framework for Robust SLAM in Dynamic Environments, which applies Bayesian filtering and multi-view probabilistic updates directly to the Gaussians, allowing the system to naturally prune dynamic artifacts and maintain stable tracking. To leverage this dense representation for localization, [Gennady Sidorov](/author/gennady-sidorov) et al. present GSplatLoc: Grounding Keypoint Descriptors into 3D Gaussian Splatting for Improved Visual Localization, grounding keypoint descriptors into the 3DGS map for robust pose refinement.
Furthermore, 3DGS can be used to simulate dynamic variations to train better localization models. [Sihang Li](/author/sihang-li-xhqt) et al. develop Adversarial Exploitation of Data Diversity Improves Visual Localization, generating diverse synthetic training views from 3DGS models and using adversarial training to make localization networks inherently robust to changing weather and dynamic driving conditions.
Continuous-Time Trajectories for High-Speed Dynamics
Handling dynamic environments often coincides with aggressive or rapid sensor motion, where discrete-time state estimation fails due to synchronization mismatches between sensors.
Jian Sun et al. address this in CT-UIO: Continuous-Time UWB-Inertial-Odometer Localization Using Non-Uniform B-spline with Fewer Anchors by parameterizing the robot's trajectory as a continuous non-uniform B-spline. This allows the system to smoothly integrate asynchronous sensor data (Ultra-Wideband, IMU, and odometry) and maintain high localization accuracy during rapid linear and rotational movements, even when physical anchors are scarce.
Key Foundational Work
DM-SLAM: A Feature-Based SLAM System for Rigid Dynamic Scenes (2020) — contributed 1 concept elements to this direction.
Stable Stereo based EKF-SLAM in Dynamic Situation (2015) — contributed 1 concept elements to this direction.
Multi-object Monocular SLAM for Dynamic Environments (2020) — contributed 1 concept elements to this direction.