Introduction
Multi-Agent Reinforcement Learning (MARL) relies heavily on communication to coordinate complex tasks, from warehouse robotics to drone swarm navigation. However, these communication channels are vulnerable "soft targets." They are susceptible to both random environmental noise and deliberate adversarial attacks, where malicious agents send deceptive data to induce system failure.
Traditional approaches to this problem typically employ passive defense strategies. These methods attempt to train policies that are inherently robust to a certain level of data corruption. However, they often suffer from a "garbage in, garbage out" limitation: if the input is sufficiently corrupted, the policy inevitably fails. Furthermore, passive robustness often comes at the cost of performance in clean, non-adversarial environments.
The ADMAC Framework
To address these vulnerabilities, researchers have introduced the Active Defense Multi-Agent Communication (ADMAC) framework. Unlike passive systems that treat all incoming messages as equally valid, ADMAC enables agents to "fact-check" communications against their own local observations.
ADMAC is built upon two core technical pillars:
- Reliability Estimator: A classifier that evaluates each incoming message. By comparing a message (e.g., "the target is at coordinates X, Y") against the agent's local observations and internal memory (hidden states), it assigns a reliability weight between 0 and 1. This allows the agent to identify and down-weight messages that contradict its own "local truth."
- Decomposable Message Aggregation Policy Net: Standard neural architectures often aggregate input data in a way that makes it difficult to isolate individual corrupted signals. ADMAC utilizes a decomposable structure where the final decision is a summation of a "base preference" (derived from local data) and several "message-specific preferences." The reliability weight directly scales the impact of each specific message on the final action distribution.
Training and Evaluation
The ADMAC training pipeline involves three stages: first, training the policy in a clean environment to establish a gold standard for cooperation; second, generating a dataset by subjecting these agents to various attacks and labeling messages as "reliable" or "unreliable" based on their utility; and third, training the Reliability Estimator using supervised learning on this labeled data.
Performance was evaluated across three communication-critical environments: Food Collector, Predator Prey, and Treasure Hunt. These environments were subjected to diverse attack types, including Gaussian noise and sophisticated adversarial methods like Fast Gradient Sign Method (FGSM) and Projected Gradient Descent (PGD).
Key Results and Insights
ADMAC demonstrates superior robustness compared to existing baselines like TARMAC (no defense) and AME (consensus-based defense). A critical finding is ADMAC's resilience to high-intensity attacks.
As shown in the experimental results, even when 30% of messages are maliciously corrupted ($p=0.3$), ADMAC's performance remains stable. In contrast, other methods see their completion times skyrocket. Furthermore, the reliability estimator achieved high precision and recall (e.g., 0.90/0.86 in Food Collector), proving that agents can effectively learn to identify deceptive communications without becoming overly "paranoid" and ignoring useful information.
Conclusion
ADMAC represents a significant step toward secure and robust multi-agent systems. By shifting the defensive burden from the policy's general "toughness" to an active reliability judgment, the framework maintains high performance in both peaceful and adversarial settings. Future work will focus on extending this framework to continuous control spaces and improving scalability for larger agent swarms where the computational overhead of individual message judgment may become a factor.
Summary updated