Divide and Conquer: Provably Unveiling the Pareto Front with Multi-Objective Reinforcement Learning

Willem Röpke, Mathieu Reymond, Patrick Mannion, Diederik M Roijers, Ann Nowe, Roxana Rădulescu

Abstract

A notable challenge in multi-objective reinforcement learning is obtaining a Pareto front of policies to attain optimal performance under different preferences. We introduce Iterated Pareto Referent Optimisation (IPRO), which decomposes finding the Pareto front into a sequence of constrained single-objective problems. This enables us to guarantee convergence while providing an upper bound on the distance to undiscovered Pareto optimal solutions at each step. Empirical evaluations demonstrate that IPRO matches or outperforms methods that require additional assumptions. Furthermore, IPRO is a general-purpose multi-objective optimisation method, making it applicable to domains beyond reinforcement learning.
BibTeX
@inproceedings{ropke2025divide,
  author = {Willem Röpke and Mathieu Reymond and Patrick Mannion and Diederik M Roijers and Ann Nowe and Roxana Rădulescu},
  title = {{Divide and Conquer: Provably Unveiling the Pareto Front with Multi-Objective Reinforcement Learning}},
  booktitle = {AAMAS},
  year = {2025},
}
Figure