TACTIC: Task-Agnostic Contrastive pre-Training for Inter-Agent Communication

Peihong Yu, Manav Mishra, Syed Zaidi, Pratap Tokekar

Introduction

In Multi-Agent Reinforcement Learning (MARL), effective coordination often hinges on the ability of individual agents to share information. However, agents typically operate under partial observability, constrained by a specific "sight range." This creates a fundamental trade-off: a limited sight range provides insufficient context for coordination, while an overly expansive sight range leads to information overload, hindering the learning process. Furthermore, existing communication protocols are often brittle, failing when the visibility conditions during deployment differ from those encountered during training.

This paper introduces TACTIC (Task-Agnostic Contrastive pre-Training for Inter-Agent Communication), a novel framework designed to decouple the learning of communication protocols from task-specific policy optimization. By utilizing contrastive learning, TACTIC enables agents to develop a robust, generalizable "language" that allows them to effectively "see" through their teammates, even under varying visibility conditions.

The Sight Range Dilemma

Most current MARL models assume that the environment's visibility parameters remain static between training and execution. In real-world applications—such as autonomous vehicles navigating changing weather or drones operating in smoke-filled environments—this assumption is frequently violated. When sight ranges fluctuate, traditional communication models often fail because their internal representations are overfitted to the specific visibility levels seen during training. TACTIC addresses this by training agents to reconstruct a global egocentric view from local observations and received messages, making the communication protocol resilient to changes in sensor range.

Methodology

TACTIC employs a two-stage learning process: offline task-agnostic pre-training followed by online policy integration.

1. Offline Contrastive Pre-training

Before agents are tasked with specific objectives, they undergo a pre-training phase using an offline dataset of trajectories. This phase focuses on learning a communication module through several key objectives:

  • Global Information Alignment (GIA): This objective ensures that features generated by different agents are consistent and grounded in the global state of the environment.
  • Feature Integration Alignment (FIA): This is the core innovation of TACTIC. It forces an agent to align its integrated representation—composed of its local observation and incoming messages—with its "full egocentric state." This full state represents what the agent would see if it had an unlimited sight range. By matching these representations, the agent learns to use communication to fill in the gaps in its own vision.
  • Auxiliary Losses: To further ground the representations, the model incorporates a reconstruction loss (ensuring the features can describe the environment) and a dynamics loss (ensuring the model understands environmental transitions over time).

2. Online Policy Integration

Once the communication protocol is established, the communication modules are frozen. The agents then utilize these fixed "translation" tools to learn task-specific policies using standard MARL algorithms, such as QMIX. Because the communication is task-agnostic, the learned protocol can be reused across different tasks within the same environment without retraining.

Experimental Setup

The researchers evaluated TACTIC using the SMACv2 (StarCraft Multi-Agent Challenge) benchmark. SMACv2 is significantly more challenging than its predecessor due to procedural generation, which prevents agents from memorizing specific paths, and strictly enforced partial observability. Tests were conducted across Terran, Protoss, and Zerg maps with varying team sizes (5, 10, and 20 agents).

Key Results and Insights

Superior Generalization

TACTIC demonstrated remarkable robustness in "heat-map" evaluations, where agents were tested on sight ranges different from those used in training. While baseline models like QMIX-Att and TarMAC saw performance collapse when visibility deviated from training conditions, TACTIC maintained high win rates. For example, on the Terran 5v5 map, agents trained with a minimal sight range (0.2 ratio) but tested with a maximum range (5.0 ratio) achieved a 0.41 win rate with TACTIC, compared to less than 0.10 for all baselines.

Training Efficiency

TACTIC converged faster than or equal to state-of-the-art methods in most scenarios. It proved particularly dominant in low-visibility training environments, where other models often failed to learn any viable coordination strategies.

Data Flexibility

An important finding was that TACTIC does not require expert demonstrations for pre-training. The authors observed that the communication protocol could be learned effectively using random exploration data. This suggests that the model primarily needs diverse examples of environmental perspectives rather than high-quality task performance to learn how to communicate.

Limitations and Future Work

While TACTIC offers significant improvements in robustness, some limitations remain. Currently, the sight range is fixed during the execution of a single episode; future research could investigate dynamic, real-time fluctuations in visibility. Additionally, the reliance on an offline dataset—even a random one—is a prerequisite that may not always be feasible. Finally, exploring TACTIC's performance under extreme bandwidth constraints or noisy communication channels would be a valuable direction for future robotics applications.

Conclusion

TACTIC shifts the paradigm of multi-agent communication from simple data sharing to a mechanism for global state reconstruction. By leveraging contrastive pre-training, it provides a task-agnostic solution to the sight range dilemma, ensuring that multi-agent systems remain effective and coordinated even when their environmental perception is compromised.

Summary updated

Abstract

The "sight range dilemma" in cooperative Multi-Agent Reinforcement Learning (MARL) presents a significant challenge: limited observability hinders team coordination, while extensive sight ranges lead to distracted attention and reduced performance. While communication can potentially address this issue, existing methods often struggle to generalize across different sight ranges, limiting their effectiveness. We propose TACTIC, Task-Agnostic Contrastive pre-Training strategy Inter-Agent Communication. TACTIC is an adaptive communication mechanism that enhances agent coordination even when the sight range during execution is vastly different from that during training. The communication mechanism encodes messages and integrates them with local observations, generating representations grounded in the global state using contrastive learning. By learning to generate and interpret messages that capture important information about the whole environment, TACTIC enables agents to effectively "see" more through communication, regardless of their sight ranges. We comprehensively evaluate TACTIC on the SMACv2 benchmark across various scenarios with broad sight ranges. The results demonstrate that TACTIC consistently outperforms traditional state-of-the-art MARL techniques with and without communication, in terms of generalizing to sight ranges different from those seen in training, particularly in cases of extremely limited or extensive observability.
BibTeX
@inproceedings{yu2025tactic,
  author = {Peihong Yu and Manav Mishra and Syed Zaidi and Pratap Tokekar},
  title = {{TACTIC: Task-Agnostic Contrastive pre-Training for Inter-Agent Communication}},
  booktitle = {MALTA @ AAAI},
  year = {2025},
  url = {https://openreview.net/forum?id=CMYiApcudp},
}
Figure