Introduction
Multi-Agent Pathfinding (MAPF) is a critical challenge in robotics and logistics, requiring multiple agents to navigate to their respective targets without collisions. While centralized planners offer high performance, they often fail to scale to large numbers of agents. Conversely, decentralized Reinforcement Learning (MARL) scales effectively but frequently struggles with complex coordination, leading to "deadlocks" in congested areas like narrow corridors.
SRMT (Shared Recurrent Memory Transformer) addresses these coordination bottlenecks by introducing a "global workspace" for multi-agent systems. This approach allows decentralized agents to implicitly share information through a pooled memory buffer, enabling them to negotiate space and resolve deadlocks without explicit communication protocols.
Architecture and Methods
The core innovation of this work is the Shared Recurrent Memory Transformer (SRMT) architecture. Inspired by "Global Workspace Theory" from cognitive science, the model coordinates specialized modules by broadcasting information through a common workspace.
As shown in the architecture diagram, the process follows several key steps:
- Spatial Encoding: Each agent processes its local 2D observation (obstacles, other agents, and targets) using a ResNet-based encoder.
- Self-Attention: The agent's memory vector and recent observation history are processed through a self-attention layer to capture temporal dependencies.
- Cross-Attention & Shared Memory: The output interacts with a "Shared Memory" pool via a cross-attention layer. This pool contains the memory vectors of all agents at the current time step, providing global context.
- Implicit Communication: By attending to the shared pool, agents gain a global understanding of the system state. This allows for sophisticated coordination without the need for complex, learned message-passing protocols.
Experimental Setup and Datasets
The researchers evaluated SRMT across a variety of challenging environments designed to test both coordination logic and large-scale scalability:
- Bottleneck Task: A specialized environment with two rooms connected by a single-cell-wide corridor, designed to test deadlock resolution.
- POGEMA Benchmark: A standard suite for partially observable MAPF, including dense mazes and random grids.
- Large-Scale Maps: Evaluation included complex "Warehouse" logistics environments and "MovingAI" city maps (e.g., Berlin, Paris) to test real-world applicability.
Key Insights and Results
SRMT demonstrates several significant advantages over traditional MARL baselines:
- Zero-Shot Generalization: One of the most remarkable findings is the model's ability to generalize. After being trained on corridors up to 30 cells long, SRMT successfully navigated corridors up to 1,000 cells long without further training. This suggests the model learns general principles of yielding and turn-taking rather than specific map patterns.
- Robustness to Sparse Rewards: Unlike many RL models that fail when rewards are only given upon reaching a goal, SRMT maintains high performance. Its shared memory mechanism allows agents to coordinate effectively even when individual feedback is infrequent.
- High Throughput: In Lifelong MAPF scenarios—where agents receive new goals immediately upon reaching their current target—SRMT outperformed or matched state-of-the-art MARL and planning algorithms. It maintained high throughput even as agent counts scaled from 64 to 256 on large city maps.
Conclusion and Future Work
SRMT provides a scalable, decentralized solution for complex multi-agent coordination. By leveraging a shared latent memory, agents can "negotiate" shared spaces implicitly. While the current model assumes synchronized time steps and perfect sensing, future research will focus on extending SRMT to asynchronous environments and integrating it with advanced path-planning heuristics to further improve performance in hyper-congested industrial settings.
Summary updated