When coordinating a large team of autonomous agents—whether they are a swarm of quadrotors, a fleet of self-driving cars, or a group of robotic soccer players—centralized planning quickly becomes impossible. In a centralized system, a single controller must ingest the states of all agents, compute a joint action space, and issue commands. Because the joint state and action spaces grow exponentially with the number of agents, this approach suffers from the curse of dimensionality, leading to computational bottlenecks and a high risk of failure if communication links drop.
To build truly scalable multi-agent systems, researchers have shifted toward decentralized execution based on local observations. The core intuition is simple: instead of forcing a central controller to solve a massive global optimization problem, we can design individual agents to make independent decisions using only their immediate surroundings. By restricting each agent’s focus to a local field of view and its nearest neighbors, the computational complexity per agent remains constant, allowing the system to scale seamlessly to dozens or hundreds of agents.
Constant-Size State Spaces via Local Sensing
The primary reason decentralized local-sensing works is that it decouples an agent's decision-making complexity from the total size of the swarm. If an agent's policy only processes information within its immediate field of view, the input dimensionality remains fixed whether there are five agents in the environment or five thousand.
For instance, in dense traffic scenarios, Jiaxun Cui et al. demonstrated this concept in Scalable Multiagent Driving Policies for Reducing Traffic Congestion. Rather than relying on a global traffic management infrastructure, autonomous vehicles execute policies based exclusively on immediate neighbor data, such as relative speed and distance. This local-sensing mechanism allows the policy to scale from small simulations to massive, realistic highway models without retraining.
Similarly, in highly dynamic environments like robotic soccer, Brandon Ho et al. leveraged local observations in Towards Learning Scalable Agile Dynamic Motion Planning for Robosoccer Teams with Policy Optimization to achieve agile, real-time motion planning. By processing only local spatial data, individual agents can navigate around fast-moving opponents without waiting for a central server to calculate global trajectories.
In mixed environments containing both cooperative and non-cooperative entities, Zuxin Liu et al. developed MAPPER in MAPPER: Multi-Agent Path Planning with Evolutionary Reinforcement Learning in Mixed Dynamic Environments. By decomposing long-range goals into local waypoints and utilizing an image-based representation of nearby dynamic obstacles, the authors showed that agents can safely navigate complex spaces relying solely on independent, local observations.
Centralized Planning (Scales Exponentially):
[Global State: Agent 1, 2, ..., N] ---> [Central Controller] ---> [Joint Actions]
Decentralized Planning (Scales Linearly):
[Local Obs 1] ---> [Agent 1 Policy] ---> [Action 1]
[Local Obs 2] ---> [Agent 2 Policy] ---> [Action 2]
Resolving Local Conflicts and Collisions
Operating purely on local observations introduces a significant challenge: how do agents avoid colliding or interfering with one another if they lack a global map? The solution lies in structured local coordination mechanisms, such as neighborhood communication, spatial partitioning, or localized market mechanisms.
An early foundation for this was established by Yoshikazu Arai et al. in Collision Avoidance in Multi-Robot Environment based on Local Communication.. The authors introduced LOCISS, an infrared sensor system that allows robots to communicate directly with immediate neighbors. Combining this local communication with reinforcement learning drastically reduced the computational complexity of learning avoidance behaviors, enabling real-time navigation without global coordination.
For continuous geometric coordination, modern approaches use dynamic spatial boundaries. In DMVC-Tracker: Distributed Multi-Agent Trajectory Planning for Target Tracking Using Dynamic Buffered Voronoi and Inter-Visibility Cells, Yun-Woo Lee et al. utilized time-varying, localized "cells" to manage multi-drone target tracking. By calculating collision-free trajectories within these mathematically defined local boundaries, the drones prevent mid-air collisions and avoid occlusions without needing a central coordinator.
When explicit tasks must be distributed under high uncertainty, the planning problem can be mathematically broken down. Jesus Capitan et al. addressed this in Decentralized Multi-Robot Cooperation with Auctioned POMDPs by decomposing a massive multi-agent planning problem into individual Partially Observable Markov Decision Processes (POMDPs). The robots then use a localized auctioning mechanism to coordinate their individual policies, bypassing the exponential growth of joint state-action spaces while maintaining cooperative behavior.
Aligning Local Actions with Global Objectives
While local sensing ensures scalability, it risks making agents myopic—they might avoid immediate collisions but fail to reach their long-term, global destinations. To prevent this, successful frameworks inject low-dimensional global guidance into the local decision-making loop.
One elegant implementation of this hybrid approach is G2RL, proposed by Binyu Wang et al. in Mobile Robot Path Planning in Dynamic Environments through Globally Guided Reinforcement Learning. The authors designed a hierarchical framework where each agent conditions its policy on both its local field of view and a personalized, coarse global guidance signal. This ensures that the computational overhead does not scale with the number of agents, yet the agents do not get trapped in local minima.
[Coarse Global Path] ---> [Personalized Guidance Signal] \
---> [Agent Policy] ---> [Local Action]
[Sensor Readings] ---> [Local Field of View] /
Alternatively, agents can maintain internal approximations of their teammates' intentions. In Experimental Demonstration of Multi-Agent Learning and Planning under Uncertainty for Persistent Missions with Automated Battery Management, Nazım Kemal Üre et al. combined decentralized planning with a group aggregate variant that condenses teammate states into a compact approximation. This allows individual UAVs to execute persistent surveillance and battery management tasks autonomously while staying aligned with the team's overarching mission.
In vision-based tracking tasks, alignment can be achieved by sharing lightweight metadata. Hai Ci et al. demonstrated this in Proactive Multi-Camera Collaboration for 3D Human Pose Estimation. A team of mobile cameras makes independent movement decisions based on local visual observations and shared target metadata. By optimizing a collaborative reward function, the decentralized cameras position themselves optimally to capture 3D poses without a central director dictating their paths.
Decoupling Training from Execution
How do we train agents to act independently based only on local information while still achieving cohesive team behavior? The dominant paradigm is Centralized Training with Decentralized Execution (CTDE). During training, the learning algorithm has access to global information or joint states, allowing it to shape effective policies. Once deployed, however, the centralized training wheels are stripped away, and agents execute their learned policies purely on local observations.
For instance, in high-speed drone racing, Xian Wang et al. developed a decentralized reinforcement learning framework in Dashing for the Golden Snitch: Multi-Drone Time-Optimal Motion Planning with Multi-Agent Reinforcement Learning. By pairing a centralized training phase with a "soft" collision-free mechanism, the quadrotors learn to fly time-optimal trajectories. During real-world deployment, they rely entirely on onboard computation and local neighbor observations, achieving performance levels nearly identical to centralized optimization while remaining computationally feasible for real-time, high-speed flight.
To scale the training phase itself to even larger numbers of agents, Mengxian Li et al. introduced the GTDE framework in GTDE: Grouped Training with Decentralized Execution for Multi-agent Actor-Critic. Instead of using a single global critic that ingests all agents' states, GTDE dynamically groups agents based on their local information needs during training. This prevents the training process from bottlenecking, while ensuring that the final policies remain fully decentralized during execution.
Ultimately, this decoupling mimics natural systems. As explored by Juwon Lee et al. in Co-Evolution of Predator-Prey Ecosystems by Reinforcement Learning Agents, complex, stable, and ecologically plausible population dynamics emerge when individual predator and prey agents independently execute policies based on their local observations. By removing the requirement for global communication and centralized infrastructure, decentralized local-sensing planning provides a robust, mathematically tractable blueprint for scaling multi-agent systems to real-world deployment.
Key Foundational Work
Experimental Demonstration of Multi-Agent Learning and Planning under Uncertainty for Persistent Missions with Automated Battery Management (2012) — contributed 2 concept elements to this direction.
Collision Avoidance in Multi-Robot Environment based on Local Communication. (2001) — contributed 1 concept elements to this direction.