The Surprising Effectiveness of Visual Odometry Techniques for Embodied PointGoal Navigation

Xiaoming Zhao, Harsh Agrawal, Dhruv Batra, Alexander G. Schwing

Abstract

PointGoal navigation agents often fail when transitioning from noiseless simulations to real-world environments characterized by sensor and actuation noise. We present a robust Visual Odometry (VO) framework that utilizes geometric invariance, depth discretization, and action-specific training to bridge this gap. Our approach achieves a 71.7% success rate on the Habitat PointNav benchmark, significantly outperforming existing SLAM-based methods while operating 6.4x faster.

Introduction

PointGoal navigation is a fundamental task where an agent must navigate to a target coordinate (e.g., "5m forward, 2m right") relative to its starting position. In idealized simulation environments, agents typically have access to noise-free "GPS+Compass" sensors, leading to near-perfect success rates. However, real-world deployment introduces significant challenges: sensor noise in RGB-D cameras and actuation noise from wheel slip or floor friction. These factors degrade localization accuracy, causing standard navigation policies to fail as the agent loses track of its relative goal position.

Comparison of ideal (a) vs. realistic (b) navigation scenarios with sensor and actuation noise.

Methodology

We introduce a modernized Visual Odometry (VO) approach designed for robustness and efficiency. The core of the system is a ResNet-18 backbone that processes consecutive RGB-D observations to predict the relative transformation ($SE(2)$) between time steps. This model acts as a "drop-in" replacement for perfect localization sensors, allowing policies trained in noiseless simulations to be deployed in realistic settings.

Overview of the Visual Odometry (VO) system, including the model architecture and observation processing pipeline.

The framework incorporates several key innovations to handle noise:

  • Geometric Invariance Losses: Enforcing mathematical consistency where the transformation from frame $t$ to $t+1$ must be the inverse of the transformation from $t+1$ back to $t$.
  • Depth Discretization: Converting raw depth values into $N$ discrete one-hot channels. This makes the model less sensitive to the grainy, small-scale fluctuations common in real-world depth sensors.
  • Action-Specific Models (SepAct): Utilizing separate prediction heads for different actions (e.g., "move forward" vs. "turn left"). This accounts for the distinct noise distributions associated with different mechanical movements.
  • Soft Top-Down Projection: Projecting 3D depth data into a 2D egocentric map to provide the model with a cleaner structural representation of the environment.

Experimental Results

Evaluated on the Habitat Challenge 2020 PointNav benchmark using the Gibson-4+ dataset, our method achieved a state-of-the-art success rate of 71.7%, a significant improvement over the previous 64.5% baseline. Furthermore, the Success weighted by Path Length (SPL) increased from 37.7% to 52.5%, indicating more efficient pathfinding. Notably, our VO-based approach is 6.4x faster than previous SLAM-based methods, making it highly suitable for real-time robotics applications where computational resources are limited.

Conclusion

This work demonstrates that robust Visual Odometry can effectively replace perfect localization sensors in noisy environments. By leveraging geometric constraints and action-specific modeling, we achieve high accuracy with minimal computational overhead. Future research will focus on extending this model to non-planar $SE(3)$ motion for aerial robotics and improving performance in dynamic environments containing moving obstacles.

Summary updated

Abstract

It is fundamental for personal robots to reliably navigate to a specified goal. To study this task, PointGoal navigation has been introduced in simulated Embodied AI environments. Recent advances solve this PointGoal navigation task with near-perfect accuracy (99.6% success) in photo-realistically simulated environments, assuming noiseless egocentric vision, noiseless actuation and most importantly, perfect localization. However, under realistic noise models for visual sensors and actuation, and without access to a "GPS and Compass sensor," the 99.6%-success agents for PointGoal navigation only succeed with 0.3%. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> In this work, we demonstrate the surprising effectiveness of visual odometry for the task of PointGoal navigation in this realistic setting, i.e., with realistic noise models for perception and actuation and without access to GPS and Compass sensors. We show that integrating visual odometry techniques into navigation policies improves the state-of-the-art on the popular Habitat PointNav benchmark by a large margin, improving success from 64.5% to 71.7% while executing 6.4 times faster.
BibTeX
@misc{zhao2021surprising,
  author = {Xiaoming Zhao and Harsh Agrawal and Dhruv Batra and Alexander G. Schwing},
  title = {{The Surprising Effectiveness of Visual Odometry Techniques for Embodied PointGoal Navigation}},
  year = {2021},
  eprint = {2108.11550},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2108.11550},
}
Figure