Deep Q-Learning with Prioritized Sampling

Jianwei Zhai, Quan Liu, Zongzhang Zhang, Shan Zhong, Haijun Zhu, Peng Zhang, Cijia SunICONIP 2016 · OpenReview

Abstract

The combination of modern reinforcement learning and deep learning approaches brings significant breakthroughs to a variety of domains requiring both rich perception of high-dimensional sensory inputs and policy selection. A recent significant breakthrough in using deep neural networks as function approximators, termed Deep Q-Networks (DQN), proves to be very powerful for solving problems approaching real-world complexities such as Atari 2600 games. To remove temporal correlation between the observed transitions, DQN uses a sampling mechanism called experience reply which simply replays transitions at random from the memory buffer. However, such a mechanism does not exploit the importance of transitions in the memory buffer. In this paper, we use prioritized sampling into DQN as an alternative. Our experimental results demonstrate that DQN with prioritized sampling achieves a better performance, in terms of both average score and learning rate on four Atari 2600 games.
BibTeX
@inproceedings{zhai2016deep,
  author = {Jianwei Zhai and Quan Liu and Zongzhang Zhang and Shan Zhong and Haijun Zhu and Peng Zhang and Cijia Sun},
  title = {{Deep Q-Learning with Prioritized Sampling}},
  booktitle = {ICONIP},
  year = {2016},
  url = {https://openreview.net/forum?id=UNQ42vIXdLu},
}

Loading related content...

Figure