Learning subgoal representations from state graphs in goal-conditioned hierarchical reinforcement learning
Its authors withdrew this paper, so there is no full text to read or summarize.
Lists
Only you see this
Abstract
The integration of graphs with Goal-conditioned Hierarchical Reinforcement Learning (GCHRL) has recently gained attention, as the intermediate goals (subgoals) can be effectively sampled from graphs that naturally represent the overall task structure in most RL tasks. However, some existing approaches often rely on domain-specific knowledge to construct these graphs, limiting their applicability to new tasks. Other graph-based approaches create graphs dynamically during exploration but struggle to fully utilize them because they have problems passing the information in the graphs to newly visited states. Additionally, current GCHRL methods face challenges such as sample inefficiency and poor subgoal representations. In this paper, we present a solution to these issues through the development of a graph encoder-decoder that can evaluate unseen states. Our proposed method, Graph-Guided sub-Goal representation Generation RL (G4RL), can be incorporated into any existing GCHRL method to enhance performance. We show that the graph encoder-decoder can be effectively implemented using a network trained on the state graph generated during exploration. Empirical results indicate that leveraging high and low-level intrinsic rewards from the graph encoder-decoder significantly enhances the performance of state-of-the-art GCHRL approaches in both dense and sparse reward environments.
BibTeX
@misc{zhang2025learning,
author = {Shuyuan Zhang and Zihan Wang and Xiao-Wen Chang and Doina Precup},
title = {{Learning subgoal representations from state graphs in goal-conditioned hierarchical reinforcement learning}},
year = {2025},
url = {https://openreview.net/forum?id=1OGhJCGdcP},
howpublished = {OpenReview},
note = {Submitted to ICLR 2025},
}Loading related content...