Makf-Sr: Multi-Agent Adaptive Kalman Filtering-Based Successor Representations

Mohammad Salimibeni, Parvin Malekzadeh, Arash Mohammadi, Petros Spachos, Konstantinos N. PlataniotisICASSP 2021 · OpenReview

Abstract

The paper is motivated by the importance of the Smart Cities (SC) concept for future management of global urbanization and energy consumption. Multi-agent Reinforcement Learning (RL) is an efficient solution to utilize large amount of sensory data provided by the Internet of Things (IoT) infrastructure of the SCs for city-wide decision making and managing demand response. Conventional ModelFree (MF) and Model-Based (MB) RL algorithms, however, use a fixed reward model to learn the value function rendering their application challenging for ever changing SC environments. Successor Representations (SR)-based techniques are attractive alternatives that address this issue by learning the expected discounted future state occupancy, referred to as the SR, and the immediate reward of each state. SR-based approaches are, however, mainly developed for single agent scenarios and have not yet been extended to multi-agent settings. The paper addresses this gap and proposes the Multi-Agent Adaptive Kalman Filtering-based Successor Representation (MAKF-SR) framework. The proposed framework can adapt quickly to the changes in a multi-agent environment faster than the MF methods and with a lower computational cost compared to MB algorithms. The proposed MAKF-SR is evaluated through a comprehensive set of experiments illustrating superior performance compared to its counterparts.
BibTeX
@inproceedings{salimibeni2021makf,
  author = {Mohammad Salimibeni and Parvin Malekzadeh and Arash Mohammadi and Petros Spachos and Konstantinos N. Plataniotis},
  title = {{Makf-Sr: Multi-Agent Adaptive Kalman Filtering-Based Successor Representations}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2021},
  url = {https://openreview.net/forum?id=4QolMmOBORz},
}

Loading related content...

Figure