Automatic basis function construction for approximate dynamic programming and reinforcement learning
Lists
Only you see this
Abstract
We address the problem of automatically constructing basis functions for linear approximation of the value function of a Markov Decision Process (MDP). Our work builds on results by Bertsekas and Castañon (1989) who proposed a method for automatically aggregating states to speed up value iteration. We propose to use neighborhood component analysis (Goldberger et al., 2005), a dimensionality reduction technique created for supervised learning, in order to map a high-dimensional state space to a low-dimensional space, based on the Bellman error, or on the temporal difference (TD) error. We then place basis function in the lower-dimensional space. These are added as new features for the linear function approximator. This approach is applied to a high-dimensional inventory control problem.
BibTeX
@inproceedings{keller2006automatic,
author = {Philipp W. Keller and Shie Mannor and Doina Precup},
title = {{Automatic basis function construction for approximate dynamic programming and reinforcement learning}},
booktitle = {International Conference on Machine Learning},
year = {2006},
doi = {10.1145/1143844.1143901},
}Loading related content...