SIGNAV: Semantically-Informed GPS-Denied Navigation and Mapping in Visually-Degraded Environments

Alex Krasner, Mikhail Sizintsev, Abhinav Rajvanshi, Han-Pang Chiu, Niluthpol Mithun, Kevin Kaighn, Philip Miller, Ryan Villamil, Supun SamarasekeraWACV 2022

TL;DR

This paper introduces SIGNAV, a novel real-time semantic SLAM system designed for robust navigation and scene understanding in challenging, visually-degraded environments like dark indoor spaces. By fusing data from multiple sensors (IMU, LiDAR, cameras, wheel odometry) and employing a new 2.5D semantic segmentation technique, SIGNAV achieves comparable or superior navigation accuracy to existing state-of-the-art methods in these difficult conditions.

What Problem Does This Paper Address?

Autonomous robots often need to navigate and understand their surroundings in GPS-denied environments, such as tunnels, mines, or dark indoor spaces. While current vision-based semantic SLAM systems can provide this capability, their performance significantly degrades in visually-degraded conditions due to poor image quality. This limitation hinders critical applications like search and rescue missions. Existing LiDAR-based SLAM systems are more robust to illumination changes but struggle with geometrically repetitive environments like long corridors. This paper aims to bridge this gap by developing a system that can reliably navigate and map in these perceptually-challenging scenarios.

Methods

SIGNAV employs a multi-sensor fusion framework based on factor graphs, integrating data from four key modalities:

  • Multi-Sensor Odometry: A tightly-coupled visual-inertial odometry (VIO) system forms the core, fusing IMU and camera feature tracking. This is then loosely-coupled with LiDAR odometry (using Fast-GICP for scan-to-scan registration) and wheel odometry (formulated as a 3D velocity vector for improved robustness against vibrations). This multi-sensor approach enhances navigation accuracy and robustness by avoiding single points of failure.
  • 2.5D Semantic Segmentation: A novel technique that combines grayscale camera images and LiDAR depth maps to generate semantic labels for 3D mapped points. This method uses a mixture-of-experts architecture, fusing outputs from two pre-trained deep learning networks: one for image segmentation and another for depth map segmentation. The depth segmentation network is trained using a weakly-supervised approach, leveraging semantic outputs from the image network as pseudo ground truth. This approach is designed to be robust to varying lighting conditions and computationally efficient for embedded systems.
  • Semantically-Informed Mapping: SIGNAV refines the 3D map by identifying and removing non-rigid objects and non-Lambertian surfaces. This focuses the SLAM process on static and rigid features, improving loop detection and pose estimation accuracy.
  • 3D Mapping: This module involves loop detection (identifying previously visited places by matching key frames) and pose graph optimization. The optimized poses are then used to continuously integrate 3D LiDAR points, creating a semantic 3D map.

Datasets

The paper evaluates SIGNAV on data collected from four test scenarios within GPS-denied indoor environments. These scenarios include:

  1. Long Hallways: Under normal lighting conditions.
  2. Cubicle Office: Under normal lighting conditions, with repeated loops.
  3. Conference Rooms: In perceptually-challenging conditions, with one room totally dark and another with minimal light.
  4. Large Auditorium: In a dark environment with repeated loops.

The authors also mention collecting RGB+Depth (RGBD) data in various well-lit environments for training the depth segmentation network. No new benchmarks were introduced; existing state-of-the-art SLAM systems were used for comparison.

Key Insights and Ideas

  • Multi-sensor fusion is crucial for dark environments: Relying on a single sensor modality is insufficient. Combining visual-inertial data with LiDAR and wheel odometry provides a more robust solution than any individual sensor.
  • 2.5D semantic segmentation bridges the gap: Directly using 2D semantic segmentation fails in dark conditions, while 3D semantic segmentation is computationally expensive. The proposed 2.5D approach effectively leverages both image and depth data to provide semantic understanding even when images are poor.
  • Semantic information improves SLAM: By filtering out non-rigid and non-Lambertian features, the semantic map leads to more accurate loop detection and pose estimation, especially in static environments.
  • Low-cost sensors can be effective: The system is designed to work with affordable sensors, making it suitable for smaller autonomous robots, contrasting with high-end LiDAR systems used in some research.

Key Results

  • Navigation Accuracy: SIGNAV achieves comparable or better navigation accuracy (mean absolute position error) than state-of-the-art SLAM systems (LeGO-LOAM, LIO-SAM, CamSLAM) across all four tested scenarios, with best results in long hallways and conference rooms.
  • Robustness in Dark Environments: SIGNAV demonstrates strong performance in dark conditions (Scenarios 3 and 4), where vision-based methods like CamSLAM degrade significantly.
  • Impact of LiDAR: Incorporating LiDAR measurements (SIGNAV vs. SIGNAV (no LiDAR)) consistently improves navigation accuracy across all scenarios.
  • Impact of Semantic Segmentation: The use of 2.5D semantic segmentation improves loop detection quality by focusing on static and rigid classes, leading to slightly improved navigation accuracy.

Limitations and Future Work

  • Limitations:
    • The reliance on camera-only loop detection limits performance in Scenario 4 (dark auditorium) compared to LiDAR-based systems.
  • While robust, the system's performance in highly dynamic environments is not extensively evaluated.
  • Future Work:
    • Tightly integrate LiDAR measurements for further navigation accuracy improvements.
  • Combine LiDAR and vision for loop detection to enhance robustness.
  • Extend the system to handle more dynamic environments.

Summary updated

BibTeX
@inproceedings{krasner2022signav,
  author = {Alex Krasner and Mikhail Sizintsev and Abhinav Rajvanshi and Han-Pang Chiu and Niluthpol Mithun and Kevin Kaighn and Philip Miller and Ryan Villamil and Supun Samarasekera},
  title = {{SIGNAV: Semantically-Informed GPS-Denied Navigation and Mapping in Visually-Degraded Environments}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2022},
}
Figure