Abstract
Multi-Agent Path Finding (MAPF) is critical for large-scale logistics but faces a significant trade-off between the safety of slow classic planners and the speed of collision-prone decentralized reinforcement learning. This paper introduces PICO (PrIoritized COmmunication learning), a framework that bridges this gap. PICO learns to predict agent priorities by imitating expert planners and utilizes these priorities to establish a dynamic, cluster-based communication topology. Experimental results in dense gridworld environments demonstrate that PICO achieves higher success rates and lower collision rates compared to existing decentralized methods, while maintaining scalability to large agent populations.
Introduction
Multi-Agent Path Finding (MAPF) involves navigating multiple agents to their respective goals without collisions. This task is fundamental to automated warehouse operations. Existing solutions generally fall into two categories: classic planners like ODrM*, which are optimal and collision-free but computationally intensive, and learning-based methods like PRIMAL, which use decentralized Reinforcement Learning (RL) for fast, reactive decision-making. However, decentralized RL agents often fail in crowded spaces due to a lack of coordination, leading to "social dilemmas" where agents cannot resolve conflicts like narrow corridor passage. PICO addresses this by introducing a structured priority-based communication protocol.
Methodology
The PICO framework operates in two primary phases to resolve agent conflicts through learned coordination.
Implicit Priority Learning
Rather than using hard-coded rules, PICO employs imitation learning to predict agent priorities. By observing a classic planner (ODrM*) solve MAPF instances, the system learns to assign high priority to agents whose optimal local moves align with the expert's global coordination. Conversely, agents instructed to wait or yield are assigned lower priority.
Prioritized Communication Learning
Predicted priorities are used to structure agent interactions via a Cluster-Based Routing Protocol (CBRP). High-priority agents are designated as "Central" (C) nodes, while lower-priority agents act as "Member" (M) nodes. Members transmit their local status to their respective Central agent, which then broadcasts a coordinated strategy back to the cluster. This dynamic topology creates a "centralized-within-decentralized" architecture that balances local reactivity with group coordination.
Neural Network Architecture and Multi-Task Learning
PICO is trained using the Asynchronous Advantage Actor-Critic (A3C) algorithm. The network architecture processes both spatial features (local grid views) and non-spatial features (goal direction, priority). To enhance social awareness, the model performs multi-task learning by predicting "blocking" (whether an agent obstructs others) and "on-goal" status.
Experimental Evaluation
The framework was evaluated using a modified Asprilo benchmark for warehouse automation. Training was conducted on grids up to 70x70 with 8 agents, while testing assessed scalability on 20x20 grids with up to 64 agents and varying obstacle densities (0% to 30%).
Key Results
- Success Rate: In high-density scenarios (64 agents, 20% obstacles), PICO maintained robust performance while baselines like PRIMAL and DHC saw success rates drop near zero.
- Collision Avoidance: PICO significantly reduced collision rates. For 16 agents, PICO's collision rate was approximately 0.03, compared to 0.07-0.13 for other learning-based methods.
- Efficiency and Scalability: Although trained on only 8 agents, PICO successfully generalized to 64 agents, providing near-expert solutions much faster than classic planners.
Discussion and Conclusion
PICO's primary innovation lies in using priority as a filter for communication architecture. By designating high-priority agents as communication hubs, the system naturally replicates hierarchical decision-making. Furthermore, the inclusion of auxiliary tasks like blocking prediction encourages "selfless" behavior, such as yielding in narrow passages.
While PICO demonstrates strong generalization, it remains dependent on the quality of the expert planner used during the imitation phase. Future work may explore reducing this dependency through advanced reward shaping and extending the framework to continuous 3D environments with complex physical dynamics.
Summary updated