Generalizing Poincaré Policy Representations in Multi-agent Reinforcement Learning
Lists
Only you see this
Abstract
Learning policy representations is essential for comprehending the intricacies of agent interactions and their decision-making processes. Recent studies have found that the evolution of any state under Markov decision processes (MDPs) can be divided into multiple hierarchies based on time sequences. This conceptualization resembles a tree-growing process, where the policy and environment dynamics determine the possible branches. In this paper, the multiple agent's trajectory growing paths can be projected into a Poincaré ball, which requires the tree to grow from the origin to the boundary of the ball, deriving a new geometric idea of learning Poincaré Policy Representations (P2R) for MARL. Specifically, P2R captures the policy representation of the Poincaré ball by a hyperbolic neural network and introduces a contrast objective function that encourages embeddings of the same policy to move closer together while embeddings of different policies to move apart, which enables embed policies with low distortion. Experimental results provide empirical evidence for the effectiveness of the P2R framework in cooperative and competitive games, demonstrating the potential of Poincaré policy representations for optimizing policies in complex multi-agent environments.
BibTeX
@misc{qu2024generalizing,
author = {Bohao Qu and Xiaofeng Cao and Zhen Fang and Qing Guo and Yi Chang},
title = {{Generalizing Poincaré Policy Representations in Multi-agent Reinforcement Learning}},
year = {2024},
url = {https://openreview.net/forum?id=2DJUXmHZ2O},
howpublished = {OpenReview},
note = {Submitted to ICLR 2024},
}Loading related content...