Q -Learning with Linear Function Approximation

Francisco S. Melo, M. Isabel Ribeiro

Abstract

In this paper, we analyze the convergence of Q-learning with linear function approximation. We identify a set of conditions that implies the convergence of this method with probability 1, when a fixed learning policy is used. We discuss the differences and similarities between our results and those obtained in several related works. We also discuss the applicability of this method when a changing policy is used. Finally, we describe the applicability of this approximate method in partially observable scenarios.
BibTeX
@inproceedings{melo2007q,
  author = {Francisco S. Melo and M. Isabel Ribeiro},
  title = {{Q -Learning with Linear Function Approximation}},
  booktitle = {Conference on Learning Theory},
  year = {2007},
  doi = {10.1007/978-3-540-72927-3\_23},
}

Loading related content...

Figure