Oracles and Followers: Stackelberg Equilibria in Deep Multi-Agent Reinforcement Learning
Lists
Only you see this
Abstract
Stackelberg equilibria arise naturally in a range of popular learning problems, such as in security games or indirect mechanism design, and have received in- creasing attention in the reinforcement learning literature. We present a general framework for implementing Stackelberg equilibria search as a multi-agent RL problem, allowing a wide range of algorithmic design choices. We discuss how previous approaches can be seen as specific instantiations of this framework. As a key insight, we note that the design space allows for approaches not previously seen in the literature, for instance by leveraging multitask and meta-RL techniques for follower convergence. We propose one such approach using contextual poli- cies and evaluate it experimentally on standard benchmark domains. Finally, we illustrate the effect of adopting designs outside the borders of our framework in controlled experiments.
BibTeX
@inproceedings{gerstgrasser2023oracles,
author = {Matthias Gerstgrasser and David C. Parkes},
title = {{Oracles and Followers: Stackelberg Equilibria in Deep Multi-Agent Reinforcement Learning}},
booktitle = {International Conference on Learning Representations},
year = {2023},
url = {https://openreview.net/forum?id=Vo1MVffQED},
}Loading related content...