SRMT: Shared Memory for Multi-agent Lifelong Pathfinding

Alsu Sagirova, Yuri Kuratov, Mikhail BurtsevarXiv 2025 · DOI · arXiv

Introduction

Multi-Agent Pathfinding (MAPF) is a critical challenge in robotics and logistics, requiring multiple agents to navigate to their respective targets without collisions. While centralized planners offer high performance, they often fail to scale to large numbers of agents. Conversely, decentralized Reinforcement Learning (MARL) scales effectively but frequently struggles with complex coordination, leading to "deadlocks" in congested areas like narrow corridors.

SRMT (Shared Recurrent Memory Transformer) addresses these coordination bottlenecks by introducing a "global workspace" for multi-agent systems. This approach allows decentralized agents to implicitly share information through a pooled memory buffer, enabling them to negotiate space and resolve deadlocks without explicit communication protocols.

Architecture and Methods

The core innovation of this work is the Shared Recurrent Memory Transformer (SRMT) architecture. Inspired by "Global Workspace Theory" from cognitive science, the model coordinates specialized modules by broadcasting information through a common workspace.

The SRMT architecture features a spatial encoder, a core with self-attention, and a cross-attention layer that interacts with a shared memory pool.

As shown in the architecture diagram, the process follows several key steps:

  1. Spatial Encoding: Each agent processes its local 2D observation (obstacles, other agents, and targets) using a ResNet-based encoder.
  2. Self-Attention: The agent's memory vector and recent observation history are processed through a self-attention layer to capture temporal dependencies.
  3. Cross-Attention & Shared Memory: The output interacts with a "Shared Memory" pool via a cross-attention layer. This pool contains the memory vectors of all agents at the current time step, providing global context.
  4. Implicit Communication: By attending to the shared pool, agents gain a global understanding of the system state. This allows for sophisticated coordination without the need for complex, learned message-passing protocols.

Experimental Setup and Datasets

The researchers evaluated SRMT across a variety of challenging environments designed to test both coordination logic and large-scale scalability:

  • Bottleneck Task: A specialized environment with two rooms connected by a single-cell-wide corridor, designed to test deadlock resolution.
  • POGEMA Benchmark: A standard suite for partially observable MAPF, including dense mazes and random grids.
  • Large-Scale Maps: Evaluation included complex "Warehouse" logistics environments and "MovingAI" city maps (e.g., Berlin, Paris) to test real-world applicability.
The model was evaluated on diverse benchmarks, including structured Warehouse layouts and complex MovingAI city maps.

Key Insights and Results

SRMT demonstrates several significant advantages over traditional MARL baselines:

  • Zero-Shot Generalization: One of the most remarkable findings is the model's ability to generalize. After being trained on corridors up to 30 cells long, SRMT successfully navigated corridors up to 1,000 cells long without further training. This suggests the model learns general principles of yielding and turn-taking rather than specific map patterns.
  • Robustness to Sparse Rewards: Unlike many RL models that fail when rewards are only given upon reaching a goal, SRMT maintains high performance. Its shared memory mechanism allows agents to coordinate effectively even when individual feedback is infrequent.
  • High Throughput: In Lifelong MAPF scenarios—where agents receive new goals immediately upon reaching their current target—SRMT outperformed or matched state-of-the-art MARL and planning algorithms. It maintained high throughput even as agent counts scaled from 64 to 256 on large city maps.

Conclusion and Future Work

SRMT provides a scalable, decentralized solution for complex multi-agent coordination. By leveraging a shared latent memory, agents can "negotiate" shared spaces implicitly. While the current model assumes synchronized time steps and perfect sensing, future research will focus on extending SRMT to asynchronous environments and integrating it with advanced path-planning heuristics to further improve performance in hyper-congested industrial settings.

Summary updated

Abstract

Coordination in decentralized multi-agent reinforcement learning (MARL) necessitates that agents share information about their behavior and intentions. Existing approaches rely on communication protocols with domain or resource constraints or centralized training that poorly scales to large agent populations. We introduce the Shared Recurrent Memory Transformer (SRMT), which enables coordination through unconstrained communication. SRMT provides a global memory workspace where agents broadcast their learned working memory states and query others' memory representations to exchange information and coordinate while maintaining decentralized training and execution. We evaluate SRMT on the Partially Observable Multi-Agent Pathfinding (PO-MAPF) problem, where coordination is vital for optimal path planning and deadlock avoidance. We demonstrate that shared memory enables emergent coordination even when the reward function provides minimal or no guidance. On the specifically constructed Bottleneck task that requires negotiation, SRMT consistently outperforms communicative and memory-augmented baselines, particularly under sparse reward signals, and successfully generalizes to longer corridors unseen during training. On POGEMA maps, SRMT scales with the increasing agents' population and map size, achieving competitive performance with recent MARL, hybrid, and planning-based methods while requiring no domain-specific heuristics. These results demonstrate that a transformer with shared recurrent memory enhances coordination in decentralized multi-agent systems. The source code for training and evaluation is available on GitHub: https://github.com/Aloriosa/srmt.
BibTeX
@misc{sagirova2025srmt,
  author = {Alsu Sagirova and Yuri Kuratov and Mikhail Burtsev},
  title = {{SRMT: Shared Memory for Multi-agent Lifelong Pathfinding}},
  year = {2025},
  eprint = {2501.13200},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2501.13200},
}
Figure