Suppressing Overestimation in Q-Learning through Adversarial Behaviors
Its authors withdrew this paper, so there is no full text to read or summarize.
Lists
Only you see this
Abstract
The goal of this paper is to propose a new Q-learning algorithm with a dummy adversarial player, which is called dummy adversarial Q-learning (DAQ), that can effectively control the overestimation bias in standard Q-learning. With the dummy player, the learning can be formulated as a two-player zero-sum game. The proposed DAQ unifies several Q-learning variations to control overestimation, such as Maxmin Q-learning and Minmax Q-learning (proposed in this paper) in a single framework. The proposed DAQ is a simple but effective way to suppress the overestimation bias from subtle rewards and can be easily applied to off-the-shelf reinforcement learning algorithms. A finite-time convergence of DAQ is analyzed from an integrated perspective by adapting an adversarial Q-learning. The performance of the suggested DAQ is empirically demonstrated under various MDP environments.
BibTeX
@misc{lee2023suppressing,
author = {HyeAnn Lee and Donghwan Lee},
title = {{Suppressing Overestimation in Q-Learning through Adversarial Behaviors}},
year = {2023},
eprint = {2310.06286},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2310.06286},
}