Long-HOT: A Modular Hierarchical Approach for Long-Horizon Object Transport

Sriram Narayanan, Dinesh Jayaraman, Manmohan Chandraker

Introduction

Most current Embodied AI tasks focus on short-term goals, such as navigating to a single visible object. However, real-world robotics requires long-horizon reasoning. A robot tasked with tidying a home must explore unknown rooms, remember object locations, manage carrying capacity, and plan efficient paths over thousands of timesteps. Existing end-to-end Reinforcement Learning (RL) approaches often fail in these scenarios due to a lack of exploration guarantees, difficulty in backtracking, and the vanishing gradient problem in long-horizon planning.

The Hierarchical Transport Policy (HTP)

To address these challenges, researchers introduced the Hierarchical Transport Policy (HTP), a modular framework that decomposes the complex transport task into specialized components managed by a high-level finite-state machine.

Overview of the HTP modular framework, illustrating the interaction between the high-level controller and specialized modules.

The HTP architecture consists of several key modules:

  • Topological Graph Builder: This module constructs a map of the environment using 'nodes' that store visual features and 'closeness scores' for objects. This allows the agent to maintain a persistent memory of the scene.
  • High-Level Controller: Acting as the central 'brain,' this controller decides which behavior to trigger—Explore, Pickup[Object], or Drop—based on the current state and graph information.
  • Pick-Drop Module: This module handles the logic for approaching and interacting with objects once they are identified in the topological graph.

Strategic Exploration and Navigation

Efficient exploration is critical for finding objects in large, unknown environments. HTP utilizes a Weighted Frontier Exploration strategy. The agent identifies 'frontiers' (boundaries between explored and unexplored space) and uses a Graph Convolutional Network (GCN) to predict an Exploration Score. This score prioritizes frontiers most likely to lead to target objects, mimicking human-like search patterns.

Visualization of the topological graph and frontier selection in a complex indoor environment.

For navigation, HTP employs a hybrid approach. It uses classical motion planning ($A^*$) to traverse known space efficiently and a specialized RL-based 'Object Nav' policy for the 'last mile'—the precise approach and grasping of an object. This combination leverages the reliability of geometric planning with the semantic flexibility of RL.

Experimental Evaluation

The framework was evaluated using the Habitat simulator with Matterport3D (MP3D) scans. The researchers introduced the Long-HOT benchmark, where an agent must transport $K$ objects to a goal under load constraints. The benchmark also includes 'container' objects that increase the agent's carrying capacity, adding a layer of strategic decision-making. Additionally, the model was tested on the MultiOn benchmark to demonstrate its versatility.

Results and Generalization

HTP demonstrated significant performance gains over traditional end-to-end models:

  • Success Rate: On the standard Long-HOT task, HTP achieved a 59% success rate, compared to 43% for the 'NoMap' RL baseline.
  • Scalability: In large-scale environments (dimensions > 40m), the RL baseline's performance dropped to near zero (2.8%), while HTP maintained a 22% success rate, showcasing superior generalization.
  • Efficiency: HTP set a new state-of-the-art on the MultiOn benchmark, improving both success rates and path efficiency (SPL).

Conclusion

Long-HOT and the HTP framework represent a significant step toward robust long-horizon robotics. By combining modular high-level control with topological memory and strategic exploration, HTP overcomes the limitations of end-to-end RL in complex, large-scale environments. Future work will focus on improving robustness to sensor noise and handling more complex obstacles and view-dependent object detection.

Summary updated

Abstract

We aim to address key challenges in long-horizon embodied exploration and navigation by proposing a long-horizon object transport task called Long-HOT and a novel modular framework for temporally extended navigation. Agents in Long-HOT need to efficiently find and pick up target objects that are scattered in the environment, carry them to a goal location with load constraints, and optionally have access to a container. We propose a modular topological graph-based transport policy (HTP) that explores efficiently with the help of weighted frontiers. Our hierarchical approach uses a combination of motion planning algorithms to reach point goals within explored locations and object navigation policies for moving towards semantic targets at unknown locations. Experiments on both our proposed Habitat transport task and on MultiOn benchmarks show that our method outperforms baselines and prior works. Further, we analyze the agent’s behavior for the usage of the container and demonstrate meaningful generalization to harder transport scenes with training only on simpler versions of the task.
BibTeX
@inproceedings{narayanan2024long,
  author = {Sriram Narayanan and Dinesh Jayaraman and Manmohan Chandraker},
  title = {{Long-HOT: A Modular Hierarchical Approach for Long-Horizon Object Transport}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2024},
  doi = {10.1109/icra57147.2024.10611192},
}
Figure