Deep Q-Network with Proximal Iteration

Kavosh Asadi, Rasool Fakoor, Omer Gottesman, Michael Littman, Alex SmolaICLR 2022 · OpenReview

Abstract

We employ Proximal Iteration for value-function optimization in reinforcement learning. Proximal Iteration is a computationally efficient technique that enables us to bias the optimization procedure towards more desirable solutions. As a concrete application of Proximal Iteration in deep reinforcement learning, we endow the objective function of the Deep Q-Network (DQN) agent with a proximal term to ensure that the online-network component of DQN remains in the vicinity of the target network. The resultant agent, which we call DQN with Proximal Iteration, or DQNPro, exhibits significant improvements over the original DQN on the Atari benchmark. Our results accentuate the power of employing sound optimization techniques for deep reinforcement learning.
BibTeX
@inproceedings{asadi2022deep,
  author = {Kavosh Asadi and Rasool Fakoor and Omer Gottesman and Michael Littman and Alex Smola},
  title = {{Deep Q-Network with Proximal Iteration}},
  booktitle = {International Conference on Learning Representations},
  year = {2022},
  url = {https://openreview.net/forum?id=qfaNCudAnji},
}

Loading related content...

Figure