Reinforcement Learning of Pareto-Optimal Multiobjective Policies Using Steering

Peter Vamplew, Rustam Issabekov, Richard Dazeley, Cameron FoaleAustralasian Conference on Artificial Intelligence 2015 · OpenReview

Abstract

There has been little research into multiobjective reinforcement learning (MORL) algorithms using stochastic or non-stationary policies, even though such policies may Pareto-dominate deterministic stationary policies. One approach is steering which forms a non-stationary combination of deterministic stationary base policies. This paper presents two new steering algorithms designed for the task of learning Pareto-optimal policies. The first algorithm (w-steering) is a direct adaptation of previous approaches to steering, and therefore requires prior knowledge of recurrent states which are guaranteed to be revisited. The second algorithm (Q-steering) eliminates this requirement. Empirical results show that both algorithms perform well when given knowledge of recurrent states, but that Q-steering provides substantial performance improvements over w-steering when this knowledge is not available.
BibTeX
@inproceedings{vamplew2015reinforcement,
  author = {Peter Vamplew and Rustam Issabekov and Richard Dazeley and Cameron Foale},
  title = {{Reinforcement Learning of Pareto-Optimal Multiobjective Policies Using Steering}},
  booktitle = {Australasian Conference on Artificial Intelligence},
  year = {2015},
  url = {https://openreview.net/forum?id=1SYfCiZqL4q},
}

Loading related content...

Figure