Multi-objective χ-Armed bandits

Kristof Van Moffaert, Kevin Van Vaerenbergh, Peter Vrancx, Ann Nowé

Abstract

Many of the standard optimization algorithms focus on optimizing a single, scalar feedback signal. However, real-life optimization problems often require a simultaneous optimization of more than one objective. In this paper, we propose a multi-objective extension to the standard χ-armed bandit problem. As the feedback signal is now vector-valued, the goal of the agent is to sample actions in the Pareto dominating area of the objective space. Therefore, we propose the multi-objective Hierarchical Optimistic Optimization strategy that discretizes the continuous action space in relation to the Pareto optimal solutions obtained in the multi-objective objective space. We experimentally validate the approach on two well-known multi-objective test functions and a simulation of a real life application, the filling phase of a wet clutch. We demonstrate that the strategy allows to identify the Pareto front after just a few epochs and to sample accordingly. After learning, several multi-objective quality indicators indicate that the set of sampled solutions by the algorithm very closely approximates the Pareto front.
BibTeX
@inproceedings{moffaert2014multi,
  author = {Kristof Van Moffaert and Kevin Van Vaerenbergh and Peter Vrancx and Ann Nowé},
  title = {{Multi-objective χ-Armed bandits}},
  booktitle = {IJCNN},
  year = {2014},
  doi = {10.1109/ijcnn.2014.6889753},
}

Loading related content...

Figure