THDA: Treasure Hunt Data Augmentation for Semantic Navigation

Oleksandr Maksymets, Vincent Cartillier, Aaron Gokaslan, Erik Wijmans, Wojciech Galuba, Stefan Lee, Dhruv Batra

Introduction

ObjectGoal navigation (ObjectNav) is a fundamental task in embodied AI, requiring an agent to locate a specific object category (e.g., "chair") within an unfamiliar environment. While PointGoal navigation, which relies on precise coordinates, is largely considered solved, ObjectNav remains a significant challenge. Recent research identifies a critical bottleneck: extreme overfitting. Instead of learning general semantic search strategies—such as "toilets are typically found in bathrooms"—agents often memorize the specific floor plans of training environments. Consequently, when deployed in novel settings, these agents fail to generalize, achieving success rates as low as 6% on standard benchmarks.

The Memorization Trap

The authors observe that standard reinforcement learning (RL) setups inadvertently encourage memorization. In a limited set of training houses, the most efficient way to maximize reward is to learn the shortest path to fixed goal locations. This "hindsight bias" results in high training performance that collapses during testing. To address this, the researchers propose a three-pronged approach designed to force agents to learn exploration rather than memorization.

Methodology: Treasure Hunt Data Augmentation

The core of the proposed solution is Treasure Hunt Data Augmentation (THDA). This technique involves taking 3D scans of common household items from the YCB dataset and procedurally "teleporting" them into random locations within high-fidelity 3D scans of real-world homes (Matterport3D).

Overview of the THDA framework, showing the insertion of YCB objects (e) into a 3D environment and the resulting navigation paths (1, 2, 3) for the agent.

By creating an effectively infinite variety of object placements, THDA prevents the agent from relying on a static map. The agent is forced to actively search for the "treasure" in each episode.

In addition to THDA, the authors introduced two supporting strategies:

  1. Sensor Simplification: To prevent the model from recognizing specific training rooms through rich visual textures, the agent's input is restricted to depth maps and a goal segmentation mask. This forces the neural network to learn more general geometric and semantic features.
  2. ExploreTillSeen Reward: Traditional RL rewards often penalize every step taken, which discourages the broad exploration necessary for ObjectNav. The authors implemented a two-stage reward: the agent is first rewarded for increasing its total "seen" area; once the target object is detected, the reward switches to incentivizing the shortest path to that object.

Experimental Results

The impact of these methods was evaluated on the Habitat benchmark using the Matterport3D dataset. The results demonstrate a substantial leap in performance:

  • Success Rate: The success rate improved from 14% to 27%, representing a 90% relative increase over previous state-of-the-art models.
  • Path Efficiency (SPL): Success weighted by Path Length (SPL) increased from 7.5 to 11.1, indicating that agents found objects using more direct routes.
  • Zero-Shot Generalization: In a striking test of robustness, models trained exclusively on augmented YCB objects achieved an 11% success rate on original ObjectNav goals they had never seen during training. This confirms that the agent learned a generalizable search strategy rather than environment-specific shortcuts.

Conclusion and Future Work

THDA demonstrates that "vanilla" neural architectures can achieve high performance in complex navigation tasks if the training data and reward structures are carefully designed to prevent overfitting. While the 27% success rate is a record-breaking achievement, it remains far from human-level performance. Future work will focus on improving semantic segmentation—a current bottleneck—and bridging the "sim-to-real" gap, as augmented objects do not always perfectly match the lighting and physics of their virtual surroundings.

Summary updated

Abstract

Can general-purpose neural models learn to navigate? For PointGoal navigation (‘go to Δx, Δy’), the answer is a clear ‘yes’ – mapless neural models composed of task-agnostic components (CNNs and RNNs) trained with large-scale model-free reinforcement learning achieve near-perfect performance [27]. However, for ObjectGoal navigation (‘find a TV’), this is an open question; one we tackle in this paper. The current best-known result on ObjectNav with general-purpose models is 6% success rate [25].First, we show that the key problem is overfitting. Large-scale training results in 94% success rate on training environments and only 8% in validation. We observe that this stems from agents memorizing environment layouts during training – sidestepping the need for exploration and directly learning shortest paths to nearby goal objects. We show that this is a natural consequence of optimizing for the task metric (which in fact penalizes exploration), is enabled by powerful observation encoders, and is possible due to the finite set of training environment configurations.Informed by our findings, we introduce Treasure Hunt Data Augmentation (THDA) to address overfitting in ObjectNav. THDA inserts 3D scans of household objects at arbitrary scene locations and uses them as ObjectNav goals – augmenting and greatly expanding the set of training layouts. Taken together with our other proposed changes, we improve the state of art on the Habitat ObjectGoal Navigation benchmark by 90% (from 14% success rate to 27%) and path efficiency by 48% (from 7.5 SPL to 11.1 SPL).
BibTeX
@inproceedings{maksymets2021thda,
  author = {Oleksandr Maksymets and Vincent Cartillier and Aaron Gokaslan and Erik Wijmans and Wojciech Galuba and Stefan Lee and Dhruv Batra},
  title = {{THDA: Treasure Hunt Data Augmentation for Semantic Navigation}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2021},
  doi = {10.1109/iccv48922.2021.01509},
}
Figure