Long-HOT: A Modular Hierarchical Approach for Long-Horizon Object Transport

Sriram Narayanan, Dinesh Jayaraman, Manmohan Chandraker

TL;DR

This paper introduces "Long-HOT," a new benchmark for long-horizon embodied object transport, and a modular hierarchical policy (HTP) to tackle it. HTP uses a topological graph to explore unknown environments and combines motion planning with learned policies for efficient object manipulation and transport, outperforming existing methods.

What Problem Does This Paper Address?

Embodied AI agents, like robots, often struggle with tasks that require planning and exploration over extended periods (long horizons). Traditional end-to-end reinforcement learning methods falter due to challenges in guaranteeing exploration, backtracking effectively, and planning complex, multi-step sequences. Specifically, transporting objects in large, unseen environments presents a significant hurdle. Agents need to not only navigate but also identify, pick up, and deliver multiple objects while potentially managing constraints like carrying capacity. Existing benchmarks often simplify these challenges, making it difficult to evaluate truly long-horizon capabilities.

Methods

The core contribution is the Hierarchical Transport Policy (HTP), a modular framework designed for long-horizon embodied tasks. HTP operates on three levels:

  • High-Level Controller: A finite state machine that decides between three high-level actions: Explore, Pickup [Object], or Drop. This decision is guided by object closeness scores, hand state, and goal state.
  • Exploration Module: This module uses a weighted frontier exploration technique. It builds a topological graph of the explored environment, where nodes represent locations and edges represent traversable paths. Each node stores features derived from RGBD images, an exploration score (FE) predicting the likelihood of finding an object, and an object closeness score (Fo) indicating proximity to objects. Frontiers (potential exploration points) are weighted based on these scores to guide efficient exploration.
  • Pick-Drop Module: This module handles object interaction. It identifies the most promising node for picking or dropping an object based on the closeness score. If the agent isn't at the target node, a motion planner guides it there. Then, an object navigation policy (learned via RL) takes over to move the agent to the semantic target object.

The approach combines learned policies for navigating to unknown semantic targets with classical motion planning (A*) for reaching known point goals within explored regions.

Datasets

  • Long-HOT: A novel benchmark environment introduced by the authors, built within the Habitat [29] simulation platform. It focuses on object transport in large, partially observed indoor scenes, requiring agents to find, pick up, and deliver multiple objects to a goal location, with optional access to a container. The benchmark includes different task levels (default, hard, harder) to modulate complexity.
  • MultiON [36]: An existing sequential multi-object navigation task used to evaluate the proposed HTP framework on a different, albeit related, challenge.

Key Insights and Ideas

  • Modularity is Key for Generalization: The paper strongly advocates for modularity in long-horizon tasks. HTP's ability to generalize to harder scenes after training on simpler ones highlights the benefit of decoupling navigation, exploration, and interaction.
  • Topological Graphs for Long-Horizon Memory: Building and maintaining a topological graph of the environment is crucial for efficient exploration and backtracking in large, unseen spaces. This graph acts as a form of memory, allowing the agent to reason about its surroundings over extended periods.
  • Weighted Frontiers for Smarter Exploration: Simply exploring frontiers isn't enough. Using predicted exploration scores (FE) to weight frontiers allows the agent to prioritize areas more likely to contain target objects, making exploration significantly more efficient.
  • Container as a Strategic Element: The optional container adds a layer of complexity, allowing for more efficient transport of multiple objects. The agent must decide when to seek and utilize it, balancing exploration for the container against direct object transport.

Key Results

  • Outperforms Baselines: HTP significantly outperforms various baselines, including flat policies (NoMap) and other hierarchical approaches (OracleMap-Waypoints), on both the Long-HOT and MultiON benchmarks across metrics like %Success, %Progress, SPL, and PPL.
  • Strong Generalization: HTP demonstrates impressive generalization capabilities. When trained on simpler Long-HOT tasks, it performs substantially better on harder, larger scenes compared to methods that degrade catastrophically. For instance, NoMap's success rate drops from 39% to as low as 2.8% on harder levels, while HTP maintains competitive performance (up to 33% and 22%).
  • Effectiveness of Components: Ablation studies show that each component of HTP (graph memory, weighted frontiers, object closeness scores) is critical for its performance. Removing the graph memory, for example, leads to a significant performance drop.
  • State-of-the-Art on MultiON: HTP achieves performance comparable to or better than existing state-of-the-art methods on the MultiON challenge.

Limitations and Future Work

  • Noiseless Odometry and Depth: The current HTP implementation assumes noiseless odometry and depth sensor data. The authors suggest that future work could relax these assumptions to build more robust methods that handle noisy inputs, similar to prior work in semantic mapping and navigation.
  • Potential for Overwriting Information: The paper notes a limitation where closeness scores for objects visible from a particular node direction might be overwritten by values obtained when the object is not in view from that specific node.
  • Failure Cases: The accompanying video showcases failure cases, including situations where the agent struggles with obstacles or ignores small frontier regions.
  • Future Work: The authors suggest exploring more complex task scenarios, such as those with end criteria similar to MultiON, which would require more sophisticated training strategies due to sparser rewards in long-horizon tasks. Further investigation into the impact of different task complexities and object goal counts is also implied.

Summary updated

Abstract

We address key challenges in long-horizon embodied exploration and navigation by proposing a new object transport task and a novel modular framework for temporally extended navigation. Our first contribution is the design of a novel Long-HOT environment focused on deep exploration and long-horizon planning where the agent is required to efficiently find and pick up target objects to be carried and dropped at a goal location, with load constraints and optional access to a container if it finds one. Further, we propose a modular hierarchical transport policy (HTP) that builds a topological graph of the scene to perform exploration with the help of weighted frontiers. Our hierarchical approach uses a combination of motion planning algorithms to reach point goals within explored locations and object navigation policies for moving towards semantic targets at unknown locations. Experiments on both our proposed Habitat transport task and on MultiOn benchmarks show that our method significantly outperforms baselines and prior works. Further, we validate the effectiveness of our modular approach for long-horizon transport by demonstrating meaningful generalization to much harder transport scenes with training only on simpler versions of the task.
BibTeX
@inproceedings{narayanan2024long,
  author = {Sriram Narayanan and Dinesh Jayaraman and Manmohan Chandraker},
  title = {{Long-HOT: A Modular Hierarchical Approach for Long-Horizon Object Transport}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2024},
  doi = {10.1109/icra57147.2024.10611192},
}
Figure