Towards Biologically-Inspired Visual SLAM in Dynamic Environments: IPL-SLAM with Instance Segmentation and Point-Line Feature Fusion

Jian Liu, Danyang Yao, Na Liu, Ye YuanBiomimetics 2025 · DOI

TL;DR

This paper introduces IPL-SLAM, a novel visual SLAM system designed for dynamic environments. By fusing instance-level object segmentation with point and line feature extraction and employing an adaptive weighting strategy, IPL-SLAM achieves significantly improved localization accuracy and robustness compared to existing methods.

What Problem Does This Paper Address?

Simultaneous Localization and Mapping (SLAM) is crucial for autonomous robots to navigate and build maps. However, the presence of dynamic objects (like people or vehicles) in real-world scenes poses a significant challenge. These moving elements introduce unreliable features that can lead to localization errors and degrade the overall performance of SLAM systems. Existing methods often struggle to effectively distinguish between static and dynamic features, leading to inaccuracies, especially in environments with significant motion.

Methods

IPL-SLAM tackles this problem by integrating several key techniques:

  • Biologically-Inspired Approach: The system draws inspiration from how biological vision systems combine semantic understanding with geometric cues for robust perception.
  • Instance Segmentation with YOLOv8: It utilizes YOLOv8 for pixel-level instance segmentation to accurately identify and delineate dynamic object regions. This provides semantic priors for subsequent filtering.
  • Point and Line Feature Fusion: IPL-SLAM extracts both traditional point features (using ORB) and line features (using LSD). This fusion provides richer geometric information than point-only methods.
  • Dynamic Feature Filtering:
    • Motion Consistency Checks: For point features, motion consistency is analyzed.
  • Angular Deviation Analysis: For line features, angular deviations are used to detect dynamic elements.
  • Contour Similarity Analysis: A sliding window approach with contour similarity is employed to determine object movement status across frames.
  • Mask Expansion: To handle edge cases where features might be on the boundary of dynamic objects, a mask expansion technique is used.
  • Adaptive-Weight Error Function: To optimize camera pose, an adaptive-weight error function is introduced. This function assigns lower weights to features that are more likely to be dynamic or exhibit inconsistencies, making the pose estimation more robust to residual dynamic elements.
  • Static Semantic Point Cloud Map: The system generates a static point cloud map, enhancing scene understanding by removing dynamic object information.

Datasets

The primary dataset used for evaluation is the TUM RGB-D dataset. Specifically, the "fr3" sequences, including "fr3/walking" (abbreviated as fr3/w/) with its sub-sequences like fr3/w/xyz, fr3/w/rpy, fr3/w/half, and fr3/w/static, were utilized.

Key Insights and Ideas

  • The Power of Fusion: Combining semantic instance segmentation with both point and line features provides a more robust representation of the environment, especially in dynamic scenarios.
  • Adaptive Weighting is Crucial: Simply filtering out dynamic objects isn't enough. Assigning adaptive weights to features based on their geometric consistency allows the system to gracefully handle residual dynamic elements and improve pose estimation accuracy.
  • Biological Inspiration Pays Off: Mimicking how biological systems integrate different cues leads to more robust and effective solutions for complex perception tasks like SLAM.
  • Line Features Matter: Incorporating line features, often overlooked in favor of points, adds valuable geometric constraints that improve accuracy, particularly in scenes with less texture.

Key Results

  • Superior Accuracy: IPL-SLAM significantly outperforms existing dynamic SLAM systems like DS-SLAM and ORB-SLAM2 in terms of Absolute Trajectory Error (ATE) and Root Mean Square Error (RMSE) on the TUM RGB-D dataset. For example, in the fr3/w/xyz sequence, IPL-SLAM achieved an RMSE of 0.0129, compared to DS-SLAM's 0.0201.
  • Robustness in Dynamic Scenes: The system demonstrates notable improvements in highly dynamic sequences (e.g., fr3/w/xyz, fr3/w/rpy), where dynamic objects are prevalent.
  • Clearer Point Clouds: IPL-SLAM generates cleaner static point cloud maps by effectively filtering out dynamic objects, avoiding the "ghosting" artifacts seen in methods like ORB-SLAM2.
  • Runtime Trade-offs: While IPL-SLAM achieves higher accuracy, its runtime is slightly higher than simpler methods due to the computational cost of instance segmentation and line feature extraction. However, the authors argue this is a worthwhile trade-off for improved performance.

Limitations and Future Work

  • Computational Cost: The current implementation's runtime is higher due to the inclusion of YOLOv8 and LSD. Future work aims to optimize this through model pruning, quantization, GPU acceleration, and exploring lightweight segmentation backbones.
  • Texture-Scarce Environments: The effectiveness of the dynamic suppression weighting scheme might be reduced in environments with very little texture.
  • Instance-Level Tracking: While YOLOv8 provides instance segmentation, extending this to instance-level tracking could further improve robustness.
  • Outdoor/Large-Scale Extension: The current evaluation is focused on indoor environments. Future work will explore extending IPL-SLAM to outdoor or large-scale scenarios, addressing challenges like varying illumination, long-range depth perception, and efficient loop closure.
  • Hardware Acceleration: The authors mention that using hardware acceleration like TensorRT could significantly reduce runtime.

Summary updated

Abstract

Simultaneous Localization and Mapping (SLAM) is a fundamental technique in mobile robotics, enabling autonomous navigation and environmental reconstruction. However, dynamic elements in real-world scenes-such as walking pedestrians, moving vehicles, and swinging doors-often degrade SLAM performance by introducing unreliable features that cause localization errors. In this paper, we define dynamic regions as areas in the scene containing moving objects, and dynamic features as the visual features extracted from these regions that may adversely affect localization accuracy. Inspired by biological perception strategies that integrate semantic awareness and geometric cues, we propose Instance-level Point-Line SLAM (IPL-SLAM), a robust visual SLAM framework for dynamic environments. The system employs YOLOv8-based instance segmentation to detect potential dynamic regions and construct semantic priors, while simultaneously extracting point and line features using Oriented FAST (Features from Accelerated Segment Test) and Rotated BRIEF (Binary Robust Independent Elementary Features), collectively known as ORB, and Line Segment Detector (LSD) algorithms. Motion consistency checks and angular deviation analysis are applied to filter dynamic features, and pose optimization is conducted using an adaptive-weight error function. A static semantic point cloud map is further constructed to enhance scene understanding. Experimental results on the TUM RGB-D dataset demonstrate that IPL-SLAM significantly outperforms existing dynamic SLAM systems-including DS-SLAM and ORB-SLAM2-in terms of trajectory accuracy and robustness in complex indoor environments.
BibTeX
@inproceedings{liu2025towards,
  author = {Jian Liu and Danyang Yao and Na Liu and Ye Yuan},
  title = {{Towards Biologically-Inspired Visual SLAM in Dynamic Environments: IPL-SLAM with Instance Segmentation and Point-Line Feature Fusion}},
  booktitle = {Biomimetics},
  year = {2025},
  doi = {10.3390/biomimetics10090558},
}
Figure