Online Model-Free RLSPI Algorithm for Nonlinear Discrete-Time Non-affine Systems

Yuanheng Zhu, Dongbin Zhao

Abstract

Policy iteration, as one kind of reinforcement learning methods is applied here to solve the optimal problem of nonlinear discrete-time non-affine system with continuous-state and continuous-action space. By applying action-value function or Q function, the implementation of policy iteration avoids the dependence on system dynamics. Online model-free recursive least-squares policy iteration (RLSPI) algorithm is proposed with continuous policy approximation. It is the first attempt to develop online LSPI algorithm for nonlinear discrete-time non-affine systems with continuous policy. A nonlinear discrete-time system is simulated to verify the efficiency of our algorithm.
BibTeX
@inproceedings{zhu2013online,
  author = {Yuanheng Zhu and Dongbin Zhao},
  title = {{Online Model-Free RLSPI Algorithm for Nonlinear Discrete-Time Non-affine Systems}},
  booktitle = {ICONIP},
  year = {2013},
  doi = {10.1007/978-3-642-42042-9\_31},
}

Loading related content...

Figure