Millimeter Wave Wireless Assisted Robot Navigation With Link State Classification

Mingsheng Yin, Akshaj Kumar Veldanda, Amee Trivedi, Jeff Zhang, Kai Pfeiffer, Yaqi Hu, Siddharth Garg, Elza Erkip, Ludovic Righetti, Sundeep Rangan

TL;DR

This paper introduces a novel approach to robot navigation using millimeter wave (mmWave) signals for precise localization. By classifying the "link state" (line-of-sight vs. non-line-of-sight) of mmWave signals, the robot can reliably use angle-of-arrival information to navigate, significantly improving success rates and reducing travel time in complex indoor environments.

What Problem Does This Paper Address?

Navigating in unknown indoor environments is a significant challenge for robots. While technologies like mmWave offer high-resolution sensing for localization, their effectiveness is hampered by signal reflections and blockages, leading to non-line-of-sight (NLOS) paths. Simply relying on the strongest signal's angle of arrival (AoA) can lead robots astray if the path is not direct. Existing methods struggle to reliably distinguish between direct (LOS) and indirect (NLOS) paths, especially in complex, cluttered indoor spaces. This paper aims to bridge this gap by developing a robust system that accurately classifies mmWave link states to enable more efficient and reliable robot navigation.

Methods

The proposed system employs a three-stage approach:

  1. Path Estimation: A modified low-rank tensor decomposition method is used to detect multipath components in mmWave signals and estimate their parameters, including angles of arrival (AoA), angles of departure (AoD), relative delays, and signal-to-noise ratios (SNRs). This method accounts for multiple antenna arrays and beam sweeping, common in mmWave systems.
  2. Link State Classification: A neural network is trained to classify the link state based on the estimated path parameters. Crucially, it distinguishes between four states: Line-of-Sight (LOS), First-Order Non-Line-of-Sight (NLOS), Higher-Order NLOS, and Outage. This multi-class classification is key to understanding the reliability of the signal path.
  3. Navigation Integration: The classified link state and estimated AoA are integrated into an Active Neural-SLAM module. The robot uses the mmWave-derived navigation goal (AoA) when the link state is deemed reliable (LOS or First-Order NLOS) and the SNR is above a threshold. Otherwise, it reverts to its standard exploration-based SLAM strategy.

Datasets

The research utilizes the Gibson dataset for indoor environments, which provides detailed 3D models. These models were processed using the Habitat-Sim simulator and then imported into the Remcom Wireless Insite ray-tracing tool to generate realistic mmWave propagation data at 28 GHz. This process created a novel, unified dataset that combines 3D environment models, camera and LiDAR data, and simulated mmWave wireless path estimation data. The dataset is made publicly available.

Key Insights and Ideas

  • The critical importance of multi-class link state classification: Simply distinguishing between LOS and NLOS is insufficient. Differentiating between first-order and higher-order NLOS is crucial for reliable navigation, as first-order NLOS paths, while indirect, can still provide useful directional cues.
  • Generalization to unseen environments: The trained neural network for link state classification demonstrates strong generalization capabilities, achieving 88% accuracy on environments it was not trained on. This suggests the learned features are robust to variations in indoor layouts.
  • Leveraging mmWave for navigation beyond simple LOS: The paper shows that even when the strongest path isn't direct, the type of indirect path (first-order NLOS) can still be used to guide the robot effectively, preventing it from getting stuck.
  • Synergy between wireless and visual sensing: While not fully integrated in this work, the results highlight the potential for a "digital twin" relationship between mmWave signals and visual data, offering rich opportunities for joint processing.

Key Results

  • The proposed "AoA when LOS or First-order NLOS" navigation algorithm achieved the highest arrival success rate across easy, moderate, and hard difficulty levels, outperforming a state-of-the-art computer vision-based navigation algorithm.
  • In moderate and hard environments, this algorithm also demonstrated superior performance in terms of arrival time relative to a baseline that knows the target's location.
  • The link state classification achieved 88% accuracy on unseen environments.
  • The AoA error for LOS paths was less than 5° for 98% of cases, and for first-order NLOS paths for 85% of cases, indicating high reliability for these states. Higher-order NLOS paths showed significantly larger AoA errors.

Limitations and Future Work

  • Joint processing of wireless and visual data: Currently, the system uses either wireless or visual information for navigation, not jointly. Future work could explore integrating these sensor modalities for a more comprehensive understanding of the environment.
  • Wireless information for SLAM: The current work uses mmWave for localization and navigation goals, but not directly for building the map within the SLAM framework. Incorporating mmWave data into SLAM is a promising avenue.
  • Availability of TX angle information: The performance of the link state classifier is slightly degraded when TX angle information is unavailable. Future work could focus on improving localization without this information.
  • Real-world deployment: While simulations are extensive, real-world validation with physical robots and mmWave hardware would be the next step.

Summary updated

Abstract

The millimeter wave (mmWave) bands have attracted considerable attention for high precision localization applications due to the ability to capture high angular and temporal resolution measurements. This paper explores mmWave-based positioning for a target localization problem where a fixed target broadcasts mmWave signals and a mobile robotic agent attempts to capture the signals to locate and navigate to the target. A three-stage procedure is proposed: First, the mobile agent uses tensor decomposition methods to detect the multipath channel components and estimate their parameters. Second, a machine-learning trained classifier is then used to predict the link state, meaning if the strongest path is line-of-sight (LOS) or non-LOS (NLOS). For the NLOS case, the link state predictor also determines if the strongest path arrived via one or more reflections. Third, based on the link state, the agent either follows the estimated angles or uses computer vision or other sensor to explore and map the environment. The method is demonstrated on a large dataset of indoor environments supplemented with ray tracing to simulate the wireless propagation. The path estimation and link state classification are also integrated into a state-of-the-art neural simultaneous localization and mapping (SLAM) module to augment camera and LIDAR-based navigation. It is shown that the link state classifier can successfully generalize to completely new environments outside the training set. In addition, the neural-SLAM module with the wireless path estimation and link state classifier provides rapid navigation to the target, close to a baseline that knows the target location.
BibTeX
@article{yin2022millimeter,
  author = {Mingsheng Yin and Akshaj Kumar Veldanda and Amee Trivedi and Jeff Zhang and Kai Pfeiffer and Yaqi Hu and Siddharth Garg and Elza Erkip and Ludovic Righetti and Sundeep Rangan},
  title = {{Millimeter Wave Wireless Assisted Robot Navigation With Link State Classification}},
  journal = {IEEE Open J. Commun. Soc.},
  year = {2022},
  doi = {10.1109/ojcoms.2022.3155572},
}
Figure