The Surprising Effectiveness of Visual Odometry Techniques for Embodied PointGoal Navigation

Xiaoming Zhao, Harsh Agrawal, Dhruv Batra, Alexander Schwing

TL;DR

This paper demonstrates that simple, robust visual odometry (VO) techniques can significantly outperform complex SLAM-based methods for robot navigation in realistic, noisy environments. By incorporating geometric invariances and robust data representations, the authors achieved a new state-of-the-art on the Habitat PointNav benchmark, improving success rates from 64.5% to 71.7% while running over six times faster.

What Problem Does This Paper Address?

PointGoal navigation—the task of moving a robot to a relative coordinate (e.g., "5m north, 3m west")—is considered "solved" in idealized simulations, reaching 99.6% success. However, these simulations often assume perfect sensors, perfect movement, and perfect localization (an "indoor GPS").

In the real world, robots face significant challenges:

  1. Sensor Noise: RGB-D cameras produce grainy or inaccurate images.
  2. Actuation Noise: Wheels slip, and a commanded 30° turn might actually result in 28° or 32°.
  3. No GPS: Indoor environments lack reliable global positioning, forcing robots to rely on their own observations to estimate their location.

Under these realistic conditions, the performance of "perfect" agents collapses to a mere 0.3% success rate. This paper seeks to bridge that gap using efficient visual odometry.

Figure 1: Noiseless (a) and noisy (b) PointGoal navigation. In the noisy setting, the agent observes: 1) sensor noises in egocentric observation; 2) actuation perturbations. The second column shows a histogram of orientation angle changes caused by a turn left action; 3) no localization information. The agent’s inaccurate localization results in uncertainty about the goal location.

Methods

The authors propose a robust Visual Odometry (VO) model based on a ResNet-18 architecture that estimates relative motion ($SE(2)$ transformation) between consecutive RGB-D frames. Their approach relies on several key technical contributions:

  • Geometric Invariance Losses: They introduce self-supervised losses based on the principle that motion should be consistent. For example, if a robot moves from point A to B and then back to A, the net estimated transformation should be zero. They derive specific rotation and translation invariance losses to enforce this.
  • Robustness to Uncertainty:
    • Dropout Ensembling: To handle noisy inputs, they use Dropout in the final layers during training and a scaled version during inference. This effectively mimics an ensemble of models to provide a more stable estimate without the computational cost of multiple forward passes.
  • Depth Discretization: Instead of using raw continuous depth values, they discretize depth into $N$ channels using one-hot encoding. This makes the model less sensitive to small, high-frequency sensor noise.
  • Soft Top-Down Projection: They project 3D point clouds from depth maps into a 2D top-down view. To combat noise that creates "holes" or "blockages" in raw projections, they use a normalized soft projection that creates a smoother, more reliable local map.
  • Action-Specific Models (SepAct): Recognizing that the distribution of motion is very different for moving forward versus turning, they train separate VO models for each action type.
Figure 2: Overview of the proposed VO method, showing how consecutive observations are processed through the ResNet-18 backbone to estimate the transformation.

Datasets

  • Habitat Simulator: The primary platform for all experiments and evaluations.
  • Gibson-4+ Dataset: Used for training (72 scenes) and validation (14 scenes).
  • Habitat Challenge PointNav Benchmark 2020: Used for final online leaderboard comparisons against other state-of-the-art methods.
  • KITTI: Used as a secondary benchmark to test the VO module's performance in outdoor driving scenarios.

Key Insights and Ideas

  • VO as a "Drop-in" Replacement: A major advantage of this approach is that a well-trained VO model can replace a perfect GPS+Compass sensor in a pre-existing navigation policy. This allows researchers to reuse complex policies trained over billions of frames without expensive retraining.
  • Simplicity Wins: While prior SOTA methods relied on complex particle-filter-based SLAM systems, this paper shows that a robustly trained, high-speed VO model can be more effective and much more efficient.
  • The Power of Soft Projections: Raw depth scatter plots are highly sensitive to noise. By normalizing the projection into a "soft" map, the agent gains a representation that is much more consistent across noisy and noiseless scenarios.
Figure 3: Steps to infer an egocentric top-down projection from depth. Top and bottom rows show inferred top-down projections from noisy and noiseless depth image at the same location. (b,e): top-down scatter plot. (c,f): the soft top-down projection. As can be seen, after processing, (c) and (f) share more similarities than (b) and (e), making the representation more robust to depth noises.

Key Results

  • New State-of-the-Art: Achieved a 71.7% success rate on the Habitat PointNav benchmark, a significant jump from the previous best of 64.5%.
  • Efficiency: The VO-based policy executes 6.4 times faster than the previous particle SLAM-based SOTA (5.83 hours vs. 37.5 hours for evaluation).
  • Improved Path Quality: Success Weighted by Path Length (SPL) improved from 37.7% to 52.5%, indicating the agent takes much more direct routes to the goal.
  • Ablation Success: The authors proved that every component—RGB data, depth discretization, invariance losses, and action-specific modeling—contributes meaningfully to the final performance.
Figure 5: Qualitative results showing the actual path (blue) vs. the agent's estimated path (red) using the VO model. The model remains accurate even through collisions and tight corners.

Limitations and Future Work

  • Planar Motion Assumption: The current model assumes motion in the 2D plane ($SE(2)$). While sufficient for most indoor tasks, it would need to be extended to $SE(3)$ for robots operating on uneven terrain or stairs.
  • Visual Dependency: Like all visual methods, it may struggle in extremely low-light or featureless environments (e.g., long, plain white corridors).
  • Integration: While the "drop-in" nature is a strength, the authors suggest that even tighter end-to-end integration between the VO module and the navigation policy could yield further gains.

Summary updated

Abstract

It is fundamental for personal robots to reliably navigate to a specified goal. To study this task, PointGoal navigation has been introduced in simulated Embodied AI environments. Recent advances solve this PointGoal navigation task with near-perfect accuracy (99.6% success) in photo-realistically simulated environments, assuming noiseless egocentric vision, noiseless actuation, and most importantly, perfect localization. However, under realistic noise models for visual sensors and actuation, and without access to a "GPS and Compass sensor," the 99.6%-success agents for PointGoal navigation only succeed with 0.3%. In this work, we demonstrate the surprising effectiveness of visual odometry for the task of PointGoal navigation in this realistic setting, i.e., with realistic noise models for perception and actuation and without access to GPS and Compass sensors. We show that integrating visual odometry techniques into navigation policies improves the state-of-the-art on the popular Habitat PointNav benchmark by a large margin, improving success from 64.5% to 71.7% while executing 6.4 times faster.
BibTeX
@misc{zhao2021surprising,
  author = {Xiaoming Zhao and Harsh Agrawal and Dhruv Batra and Alexander Schwing},
  title = {{The Surprising Effectiveness of Visual Odometry Techniques for Embodied PointGoal Navigation}},
  year = {2021},
  eprint = {2108.11550},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2108.11550},
}
Figure