Interference-Aware K-Step Reachable Communication in Multi-Agent Reinforcement Learning

Ziyu Cheng, Jinsheng Ren, Zhouxian Jiang, Chenzhihang Li, Rongye Shi, Bin Liang, Jun Yang

Abstract

Effective communication is pivotal for addressing complex collaborative tasks in multi-agent reinforcement learning (MARL). Yet, limited communication bandwidth and dynamic, intricate environmental topologies present significant challenges in identifying high-value communication partners. Agents must consequently select collaborators under uncertainty, lacking a priori knowledge of which partners can deliver task-critical information. To this end, we propose Interference-Aware K-Step Reachable Communication (IA-KRC), a novel framework that enhances cooperation via two core components: (1) a K-Step reachability protocol that confines message passing to physically accessible neighbors, and (2) an interference-prediction module that optimizes partner choice by minimizing interference while maximizing utility. Compared to existing methods, IA-KRC enables substantially more persistent and efficient cooperation despite environmental interference. Comprehensive evaluations confirm that IA-KRC achieves superior performance compared to state-of-the-art baselines, while demonstrating enhanced robustness and scalability in complex topological and highly dynamic multi-agent scenarios.

Introduction: The Challenge of Communication in Complex Topologies

In multi-agent reinforcement learning (MARL), choosing the right teammates to communicate with is often more critical than the content of the message itself. While effective communication is the backbone of cooperation, agents cannot communicate with all peers simultaneously due to bandwidth and scalability constraints. Most current systems rely on simplistic metrics to select communication partners, such as Euclidean distance (straight-line proximity) or basic line-of-sight.

Challenges in MARL communication: (a) physical barriers, (b) reachability constraints, and (c) dynamic interference zones.

These metrics often fail in complex real-world environments. For instance, two agents might be spatially "close" on a map but separated by a physical barrier, rendering cooperation impossible. Furthermore, even if agents are physically close, dynamic "interference" zones—such as areas under heavy adversarial fire—can make coordination too costly or risky. Without accounting for these topological and dynamic hurdles, teams often suffer from the "avalanche effect," where isolated agents are picked off sequentially, leading to total team collapse.

The IA-KRC Framework

The proposed Interference-Aware K-Step Reachable Communication (IA-KRC) framework addresses these challenges through two primary modules that optimize partner selection:

The IA-KRC framework architecture, integrating reachability and interference prediction for optimized agent grouping.
  1. K-Step Reachability Module: Rather than relying on straight-line distance, this module calculates the "shortest transition distance," representing the expected number of steps required to reach another agent. It restricts communication to a neighborhood of agents reachable within $K$ steps, effectively accounting for walls and obstacles.
  2. Interference Prediction Module: This module evaluates the cost of cooperation by predicting the intent of adversarial agents. It generates a directional interference potential field, creating a "cone of risk" based on an enemy's likely attack vector, allowing agents to avoid high-interference partners.

To maintain computational efficiency, IA-KRC utilizes a Multi-Layer Map that decouples environmental information based on its rate of change: a Geometric Layer for static obstacles, a Regulation Layer for rule-based changes (e.g., doors), and an Interference Layer for dynamic agent positions and attack risks. Finally, agents are organized into a Leader-Follower structure, where leaders are elected based on their number of reachable neighbors to ensure team cohesion.

Experimental Evaluation and Results

Researchers evaluated IA-KRC using the StarCraft Multi-Agent Challenge (SMACv2). To test the framework's robustness, they developed a self-play framework and designed custom, topologically challenging maps, including a Dense-Obstacle Map and a Maze-Structure Map.

IA-KRC significantly outperformed state-of-the-art baselines such as MAPPO, QMIX, and CommFormer:

  • Win Rate Dominance: On complex maps, IA-KRC achieved win rate advantages ranging from 4.58x to 31.56x over baselines.
  • Robustness and Scalability: In the Maze-Structure map, IA-KRC maintained a win rate over 81%, while other methods struggled to exceed 70%. The system also maintained high performance as team sizes scaled from 3v3 to 18v18.
  • Efficiency: Despite its sophisticated modeling, IA-KRC only added approximately 19% to training time compared to the baseline, whereas the "heavyweight" CommFormer required 4x longer to train.

Conclusion

IA-KRC demonstrates that reachability and interference awareness are superior metrics for grounding multi-agent communication in complex environments. By maintaining team "algebraic connectivity," the system prevents cascading failures common in other MARL algorithms. Future work will explore the framework's application to heterogeneous teams and more sophisticated adversarial intent prediction.

Summary updated

Abstract

Effective communication is pivotal for addressing complex collaborative tasks in multi-agent reinforcement learning (MARL). Yet, limited communication bandwidth and dynamic, intricate environmental topologies present significant challenges in identifying high-value communication partners. Agents must consequently select collaborators under uncertainty, lacking a priori knowledge of which partners can deliver task-critical information. To this end, we propose Interference-Aware K-Step Reachable Communication (IA-KRC), a novel framework that enhances cooperation via two core components: (1) a K-Step reachability protocol that confines message passing to physically accessible neighbors, and (2) an interference-prediction module that optimizes partner choice by minimizing interference while maximizing utility. Compared to existing methods, IA-KRC enables substantially more persistent and efficient cooperation despite environmental interference. Comprehensive evaluations confirm that IA-KRC achieves superior performance compared to state-of-the-art baselines, while demonstrating enhanced robustness and scalability in complex topological and highly dynamic multi-agent scenarios.
BibTeX
@misc{cheng2026interference,
  author = {Ziyu Cheng and Jinsheng Ren and Zhouxian Jiang and Chenzhihang Li and Rongye Shi and Bin Liang and Jun Yang},
  title = {{Interference-Aware K-Step Reachable Communication in Multi-Agent Reinforcement Learning}},
  year = {2026},
  eprint = {2603.15054},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2603.15054},
}
Figure