Learning Efficient and Robust Multi-Agent Communication via Graph Information Bottleneck

Shifei Ding, Wei Du, Ling Ding, Lili Guo, Jian Zhang

Introduction

In Multi-Agent Reinforcement Learning (MARL), effective coordination often hinges on communication. From autonomous drone fleets to tactical units in StarCraft II, agents must share information to achieve collective goals. Graph Neural Networks (GNNs) have emerged as a powerful tool for this, modeling agents as nodes and communication channels as edges. However, GNNs are prone to aggregating irrelevant information, making them vulnerable to environmental noise and adversarial attacks. A single jammed sensor or tampered link can destabilize the entire system.

To address this, researchers at the 38th AAAI Conference (2024) introduced MAGI (Multi-Agent communication via Graph Information bottleneck). This framework leverages information theory to ensure communication is both efficient and robust.

The Challenge: Filtering the Noise

Most Multi-Agent Communicative Reinforcement Learning (MACRL) methods prioritize how to pass messages rather than what to ignore. In complex, partially observable environments, agent observations are inherently noisy. Standard GNN-based layers often propagate this noise, leading to miscoordination. Furthermore, these systems are susceptible to adversarial perturbations in agent features or the communication graph's topology. The core challenge lies in learning a protocol that is expressive enough for coordination but succinct enough to filter out "chatter" and malicious interference.

Methodology: The MAGI Framework

Minimal Sufficiency via Graph Information Bottleneck

MAGI applies the Graph Information Bottleneck (GIB) principle to multi-agent systems. The goal is to learn a "minimal sufficient" message $m$ by optimizing two information-theoretic objectives:

  1. Sufficiency: Maximize the Mutual Information (MI) between the message and the selected action ($Y$). This ensures the message contains all necessary information for task performance.
  2. Minimality: Minimize the MI between the message and the input data—specifically agent features ($H$) and graph structure ($A$). This acts as a bottleneck, squeezing out irrelevant noise and redundant data.
Information-theoretic view of optimal, irrelevant, and redundant messages.

Architecture and Implementation

MAGI operates within a Centralized Training with Decentralized Execution (CTDE) paradigm. As shown in the architecture, local observations are processed via GRU/MLP units. A Graph Attention Network (GAT) serves as the base GNN to generate message representations. These messages are then constrained by the IB Loss before being concatenated with local trajectories to inform action selection.

The MAGI architecture integrated with a value-decomposition framework.

To handle the non-i.i.d. nature of multi-agent graphs, MAGI employs a local-dependence assumption. By restricting an agent's focus to its $k$-hop neighbors, the optimization of complex MI terms becomes tractable through variational bounds.

Experimental Results

The framework was evaluated on the StarCraft II Multi-Agent Challenge (SMAC) and MAgent benchmarks.

  • Robustness to Noise: In SMAC scenarios with high Gaussian noise, MAGI maintained stable win rates while baselines like TarMAC and MAGIC experienced significant performance degradation.
  • Adversarial Defense: Under sophisticated attacks targeting the graph structure (using Projected Gradient Descent), MAGI achieved a win rate of 71.34% in the MMM3 scenario, significantly outperforming TarMAC's 45.42%.
  • Scalability: In large-scale MAgent battles involving up to 50 agents, MAGI consistently delivered higher mean rewards, demonstrating its ability to handle the increased noise of dense systems.

Conclusion

MAGI demonstrates that in multi-agent communication, "less is more." By forcing messages through an information bottleneck, the model naturally learns to prioritize task-critical signals over environmental noise and structural perturbations. This approach not only improves efficiency but also provides a robust defense against adversarial interference, marking a significant step toward reliable real-world multi-agent systems.

Reference: Ding, S., et al. (2024). Learning Efficient and Robust Multi-Agent Communication via Graph Information Bottleneck. Proceedings of the AAAI Conference on Artificial Intelligence.

Summary updated

Abstract

Efficient communication learning among agents has been shown crucial for cooperative multi-agent reinforcement learning (MARL), as it can promote the action coordination of agents and ultimately improve performance. Graph neural network (GNN) provide a general paradigm for communication learning, which consider agents and communication channels as nodes and edges in a graph, with the action selection corresponding to node labeling. Under such paradigm, an agent aggregates information from neighbor agents, which can reduce uncertainty in local decision-making and induce implicit action coordination. However, this communication paradigm is vulnerable to adversarial attacks and noise, and how to learn robust and efficient communication under perturbations has largely not been studied. To this end, this paper introduces a novel Multi-Agent communication mechanism via Graph Information bottleneck (MAGI), which can optimally balance the robustness and expressiveness of the message representation learned by agents. This communication mechanism is aim at learning the minimal sufficient message representation for an agent by maximizing the mutual information (MI) between the message representation and the selected action, and simultaneously constraining the MI between the message representation and the agent feature. Empirical results demonstrate that MAGI is more robust and efficient than state-of-the-art GNN-based MARL methods.
BibTeX
@inproceedings{ding2024learning,
  author = {Shifei Ding and Wei Du and Ling Ding and Lili Guo and Jian Zhang},
  title = {{Learning Efficient and Robust Multi-Agent Communication via Graph Information Bottleneck}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2024},
  doi = {10.1609/aaai.v38i16.29682},
}
Figure