Hierarchical Policy Gradient Algorithms

Mohammad Ghavamzadeh, Sridhar MahadevanICML 2003 · OpenReview

Abstract

Hierarchical reinforcement learning is a general framework which attempts to accelerate policy learning in large domains. On the other hand, policy gradient reinforcement learning (PGRL) methods have received recent attention as a means to solve problems with continuous state spaces. However, they suffer from slow convergence. In this paper, we combine these two approaches and propose a family of hierarchical policy gradient algorithms for problems with continuous state and/or action spaces. We also introduce a class of hierarchical hybrid algorithms, in which a group of subtasks, usually at the higher-levels of the hierarchy, are formulated as value function-based RL (VFRL) problems and the others as PGRL problems. We demonstrate the performance of our proposed algorithms using a simple taxi-fuel problem and a complex continuous state and action ship steering domain.
BibTeX
@inproceedings{ghavamzadeh2003hierarchical,
  author = {Mohammad Ghavamzadeh and Sridhar Mahadevan},
  title = {{Hierarchical Policy Gradient Algorithms}},
  booktitle = {International Conference on Machine Learning},
  year = {2003},
  url = {https://openreview.net/forum?id=rJWNEo-uZr},
}

Loading related content...

Figure