Policy gradient approaches for multi-objective sequential decision making

Simone Parisi, Matteo Pirotta, Nicola Smacchia, Luca Bascetta, Marcello Restelli

Abstract

This paper investigates the use of policy gradient techniques to approximate the Pareto frontier in Multi-Objective Markov Decision Processes (MOMDPs). Despite the popularity of policy gradient algorithms and the fact that gradient ascent algorithms have been already proposed to numerically solve multi-objective optimization problems, especially in combination with multi-objective evolutionary algorithms, so far little attention has been paid to the use of gradient information to face multi-objective sequential decision problems. Two different Multi-Objective Reinforcement-Learning (MORL) approaches, called radial and Pareto following, that, starting from an initial policy, perform gradient-based policy-search procedures aimed at finding a set of non-dominated policies are here presented. Both algorithms are empirically evaluated and compared to state-of-the-art MORL algorithms on three MORL benchmark problems.
BibTeX
@inproceedings{parisi2014policy,
  author = {Simone Parisi and Matteo Pirotta and Nicola Smacchia and Luca Bascetta and Marcello Restelli},
  title = {{Policy gradient approaches for multi-objective sequential decision making}},
  booktitle = {IJCNN},
  year = {2014},
  doi = {10.1109/ijcnn.2014.6889738},
}

Loading related content...

Figure