Generalized Inverse Reinforcement Learning with Linearly Solvable MDP
Lists
Only you see this
Abstract
In this paper, we consider a generalized variant of inverse reinforcement learning (IRL) that estimates both a cost (negative reward) function and a transition probability from observed optimal behavior. In theoretical studies of standard IRL, which estimates only the cost function, it is well known that IRL involves a non-identifiable problem, i.e., the cost function cannot be determined uniquely. This problem has been solved by using a new class of Markov decision process (MDP) called a linearly solvable MDP (LMDP). In this paper, we investigate whether a non-identifiable problem occurs in the generalized variant of IRL (gIRL) using the framework of LMDP and construct a new gIRL method. The contributions of this study are summarized as follows: (i) We point out that gIRL with LMDP suffers from a non-identifiable problem. (ii) We propose a Bayesian method to escape the non-identifiable problem. (iii) We validate the proposed method by performing an experiment on synthetic data and real car probe data.
BibTeX
@inproceedings{kohjima2017generalized,
author = {Masahiro Kohjima and Tatsushi Matsubayashi and Hiroshi Sawada},
title = {{Generalized Inverse Reinforcement Learning with Linearly Solvable MDP}},
booktitle = {ECML/PKDD},
year = {2017},
doi = {10.1007/978-3-319-71246-8\_23},
}Loading related content...