Learning for multi-robot cooperation in partially observable stochastic environments with macro-actions
Lists
Only you see this
Abstract
This paper presents a data-driven approach for multi-robot coordination in partially-observable domains based on Decentralized Partially Observable Markov Decision Processes (Dec-POMDPs) and macro-actions (MAs). Dec-POMDPs provide a general framework for cooperative sequential decision making under uncertainty and MAs allow temporally extended and asynchronous action execution. To date, most methods assume the underlying Dec-POMDP model is known a priori or a full simulator is available during planning time. Previous methods which aim to address these issues suffer from local optimality and sensitivity to initial conditions. Additionally, few hardware demonstrations involving a large team of heterogeneous robots and with long planning horizons exist. This work addresses these gaps by proposing an iterative sampling based Expectation-Maximization algorithm (iSEM) to learn polices using only trajectory data containing observations, MAs, and rewards. Our experiments show the algorithm is able to achieve better solution quality than the state-of-the-art learning-based methods. We implement two variants of multi-robot Search and Rescue (SAR) domains (with and without obstacles) on hardware to demonstrate the learned policies can effectively control a team of distributed robots to cooperate in a partially observable stochastic environment.
BibTeX
@inproceedings{liu2017learning,
author = {Miao Liu and Kavinayan Sivakumar and Shayegan Omidshafiei and Christopher Amato and Jonathan P. How},
title = {{Learning for multi-robot cooperation in partially observable stochastic environments with macro-actions}},
booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
year = {2017},
doi = {10.1109/iros.2017.8206001},
}Loading related content...