Solving Deep Memory POMDPs with Recurrent Policy Gradients

Daan Wierstra, Alexander Förster, Jan Peters, Jürgen SchmidhuberICANN 2007 · OpenReview

Abstract

This paper presents Recurrent Policy Gradients, a model-free reinforcement learning (RL) method creating limited-memory sto-chastic policies for partially observable Markov decision problems (POMDPs) that require long-term memories of past observations. The approach involves approximating a policy gradient for a Recurrent Neural Network (RNN) by backpropagating return-weighted characteristic eligibilities through time. Using a “Long Short-Term Memory” architecture, we are able to outperform other RL methods on two important benchmark tasks. Furthermore, we show promising results on a complex car driving simulation task.
BibTeX
@inproceedings{wierstra2007solving,
  author = {Daan Wierstra and Alexander Förster and Jan Peters and Jürgen Schmidhuber},
  title = {{Solving Deep Memory POMDPs with Recurrent Policy Gradients}},
  booktitle = {ICANN},
  year = {2007},
  url = {https://openreview.net/forum?id=iQSnxSJlaSD},
}

Loading related content...

Figure