Maximum Entropy Semi-Supervised Inverse Reinforcement Learning

Julien Audiffren, Michal Vaľko, Alessandro Lazaric, Mohammad Ghavamzadeh

Abstract

A popular approach to apprenticeship learning (AL) is to formulate it as an inverse reinforce-ment learning (IRL) problem. The MaxEnt-IRL al-gorithm successfully integrates the maximum en-tropy principle into IRL and unlike its predecessors, it resolves the ambiguity arising from the fact that a possibly large number of policies could match the expert’s behavior. In this paper, we study an AL setting in which in addition to the expert’s tra-jectories, a number of unsupervised trajectories is available. We introduce MESSI, a novel algorithm that combines MaxEnt-IRL with principles com-ing from semi-supervised learning. In particular, MESSI integrates the unsupervised data into the MaxEnt-IRL framework using a pairwise penalty on trajectories. Empirical results in a highway driv-ing and grid-world problems indicate that MESSI is able to take advantage of the unsupervised trajecto-ries and improve the performance of MaxEnt-IRL. 1
BibTeX
@inproceedings{audiffren2015maximum,
  author = {Julien Audiffren and Michal Vaľko and Alessandro Lazaric and Mohammad Ghavamzadeh},
  title = {{Maximum Entropy Semi-Supervised Inverse Reinforcement Learning}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2015},
  url = {https://openreview.net/forum?id=ryELNVzObB},
}

Loading related content...

Figure