Multi-Agent Communication with Information Preserving Graph Contrastive Learning

Wei Du, Shifei Ding, Wei Guo, Yuqing Sun, Guoxian Yu, Lizhen CuiIJCAI 2025 · DOI

Abstract

Effective coordination in Multi-Agent Reinforcement Learning (MARL) requires robust communication protocols. Current Graph Neural Network (GNN) based communication methods often rely on the homophily assumption, assuming that physically proximate agents are the most relevant for coordination. However, in complex tasks, relevant information often comes from distant agents with similar features or from the global network structure. We present MAIL (Multi-Agent communication with Information preserving graph contrastive Learning), which utilizes Graph Contrastive Learning (GCL) to learn superior message representations. MAIL constructs three graph views—original, feature-based, and topological—and employs contrastive losses to align them, ensuring that agents preserve essential information while filtering noise. Evaluated across diverse benchmarks including StarCraft II and traffic simulations, MAIL consistently outperforms state-of-the-art GNN-based MARL methods.

Introduction

In Multi-Agent Reinforcement Learning, agents must communicate to solve cooperative tasks. GNNs are a natural choice for modeling these interactions, treating agents as nodes and communication links as edges. Despite their success, standard GNNs often struggle with the "homophily trap." They assume that neighboring nodes share similar properties, which is frequently untrue in dynamic multi-agent environments. For instance, a tactical unit in a game might need to coordinate with a distant support unit rather than its immediate neighbors.

This reliance on proximity leads to "information smearing," where irrelevant local data drowns out critical distant signals, causing coordination failure. Furthermore, standard methods often overlook feature similarity—the principle that agents performing similar roles should share similar internal representations regardless of their spatial distribution. MAIL addresses these issues by moving beyond simple physical proximity to a more holistic view of agent relationships.

The MAIL Framework

The core innovation of MAIL lies in using Graph Contrastive Learning (GCL) to refine communication. Instead of a single static graph, MAIL operates on three distinct "views" of the agent network:

  1. Original View ($v_o$): Captures immediate physical proximity (who is nearby).
  2. Agent Feature View ($v_f$): A $k$-Nearest Neighbor (kNN) graph based on the cosine similarity of agent features, connecting agents with similar roles regardless of distance.
  3. Global Topological View ($v_t$): A higher-order view capturing the broader network structure through multiple "hops."
The MAIL architecture utilizes three graph views and contrastive losses to learn robust agent representations.

These views are aligned using three contrastive loss functions: Feature Preserving Loss ($L_f$) ensures the original view retains information about feature-similar peers; Topological Preserving Loss ($L_t$) aligns local representations with the global environment structure; and Cross-Module Loss ($L_c$) maintains consistency across different encoding modules. To ensure efficiency and stability during RL training, MAIL employs Simple Graph Convolution (SGC) as its base encoder, which streamlines the message-passing process by removing unnecessary non-linearities.

Experimental Evaluation

We evaluated MAIL across four diverse benchmarks that test different aspects of coordination: Predator-Prey (grid-world coordination), Traffic Junction (collision avoidance), Battle (large-scale tactical combat), and the StarCraft Multi-Agent Challenge (SMAC) (high-complexity micromanagement).

The four evaluation environments: (a) Predator-Prey, (b) Traffic Junction, (c) Battle, and (d) SMAC.

MAIL consistently outperformed state-of-the-art baselines like MAGIC, DGN, and DICG across all scenarios:

  • Predator-Prey: MAIL reached the goal in 27.42 steps, significantly faster than the next best baseline (33.12 steps).
  • Traffic Junction: On "Hard" difficulty, MAIL achieved a 98.03% win rate, while standard methods like CommNet struggled at around 51%.
  • SMAC: In the complex MMM2 scenario, MAIL reached a 94.21% win rate, a substantial jump over the 83.45% achieved by previous leaders.

Ablation studies confirmed that the Feature Preserving module is the most critical component; its removal caused the sharpest performance drops, proving that capturing feature similarity is essential for effective multi-agent communication.

Conclusion

MAIL demonstrates that breaking the "proximity trap" through multi-view graph contrastive learning significantly enhances multi-agent coordination. By explicitly modeling feature similarity and global topology, agents can filter out irrelevant local noise and focus on meaningful coordination signals. This approach scales effectively as agent counts grow, suggesting a robust path forward for complex multi-agent systems. Future work will explore more sophisticated GCL mechanisms and the robustness of this approach in real-world, noisy communication environments.

Summary updated

BibTeX
@inproceedings{du2025multi,
  author = {Wei Du and Shifei Ding and Wei Guo and Yuqing Sun and Guoxian Yu and Lizhen Cui},
  title = {{Multi-Agent Communication with Information Preserving Graph Contrastive Learning}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.24963/ijcai.2025/8},
}
Figure