Spatial Decay of Influence in Networked Multi-Agent Systems

This direction is split into other directions: spatial correlation decay, spatial decay property. Its last version stays readable and citable here.

Version 1 of 167 papers120 conceptsfrozen 2026-10-04

In traditional Multi-Agent Reinforcement Learning (MARL), we are often haunted by the "curse of dimensionality." If you have a thousand agents, the state-action space grows exponentially, making global coordination computationally impossible. However, a growing body of research suggests that for many real-world systems—like power grids, traffic networks, and robot swarms—global coordination is actually unnecessary.

The core intuition is simple: Distant agents don't matter. This principle, often formalized as the "exponential decay of influence," allows us to solve massive global problems by looking only at small, local neighborhoods.

The Locality Principle

In a networked system, agents interact with their neighbors. While these local interactions can technically ripple across the entire network, the strength of that ripple usually dies out as it travels. David Gamarnik, David A. Goldberg, and Théophane Weber demonstrated this in Correlation Decay in Random Decision Networks, showing that decentralized agents can reach near-optimal solutions by observing only a logarithmic radius of their neighborhood.

This isn't just a heuristic; it is a structural property of networked Markov Decision Processes (MDPs). As Adam Wierman, Guannan Qu, and Na Li proved in Scalable Reinforcement Learning of Localized Policies for Multi-Agent Networked Systems, the influence of a distant agent on a local agent's long-term reward decays exponentially with the graph distance between them.

Why Influence Decays

Why does this decay happen? In physical systems, it is often due to natural "weakening" of signals. Wesley Suttle and Brian M. Sadler argue in Signal attenuation enables scalable decentralized multi-agent reinforcement learning over networks that physical signal attenuation naturally decouples distant agents, making decentralization a byproduct of physics.

In more abstract optimization problems, the decay is a mathematical property of the Bellman operator or the system's sensitivity. Patrick Rebeschini and Nicha C. Dvornek showed in Locality in Network Optimization that the sensitivity of an optimal solution to a local constraint change vanishes as you move away from that constraint. Essentially, the "correlation length" of the decision is finite.

Truncating the World

If influence decays exponentially, we can safely "truncate" our view of the world. Instead of a global value function $V(s_1, s_2, \dots, s_n)$, an agent $i$ can use a localized value function $V_i(s_{N_i})$, where $N_i$ is a small neighborhood of agents within $\kappa$ hops.

The beauty of this approach is that the error introduced by this truncation isn't just small—it's bounded. In Locally Interdependent Multi-Agent MDP, Alex DeWeese and Guannan Qu show that the performance gap between these decentralized policies and the global optimum closes exponentially fast as the visibility radius increases.

This leads to algorithms like Localized Policy Iteration (LPI) and Scalable Actor-Critic (SAC), which allow agents to learn and act using only local information. As Yiheng Lin and colleagues demonstrate in Multi-Agent Reinforcement Learning in Stochastic Networked Systems, these algorithms converge even when interactions are stochastic and non-local, provided the "$\mu$-decay" property holds.

Robustness Through Locality

Beyond scalability, locality offers a surprising benefit: robustness.

Intuitively, if you are a robot in a swarm and your policy depends on the state of every other robot, a single sensor failure on a distant peer could crash your own controller. However, if your policy is localized, you are naturally shielded from distant shocks. Wenshuai Zhao and Eetu-Aleksi Rantala found in Less Is More: Robust Robot Learning via Partially Observable Multi-Agent Reinforcement Learning that restricting agents to local observations actually makes the system more resilient to hardware failures and environmental noise than centralized controllers.

Applications at Scale

This research direction has moved from theoretical proofs to practical large-scale applications:

  • Traffic Control: Zirui Yuan and Hao Liu use local spatial reasoning in CoLLMLight to manage city-wide traffic signals with over a thousand nodes.
  • Power Grids: Feng Qiu and Tianshu Chu developed PowerNet, which uses spatial reward discounting to assign credit to agents based on their physical distance in the grid.
  • Autonomous Vehicles: Fei Miao and Songyang Han apply the locality of influence to Connected Autonomous Vehicles, enabling constant-time decision-making regardless of the total number of cars on the road.

By trusting the decay of influence, researchers have turned the "curse of dimensionality" into a "blessing of locality," proving that in a sufficiently large and connected world, you only really need to know your neighbors.

Papers in this version

Showing 60 of 67 papers.

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_spatial_decay_of_influence_in_networked__e1a3cc15,
  title = {Spatial Decay of Influence in Networked Multi-Agent Systems},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/spatial-decay-of-influence-in-networked-multi-agent-systems/rdr_e1a3cc15efbc4f69813b463223e9eed2},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_spatial_decay_of_influence_in_networked__e1a3cc15_v1,
  title = {Spatial Decay of Influence in Networked Multi-Agent Systems},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/spatial-decay-of-influence-in-networked-multi-agent-systems/rdr_e1a3cc15efbc4f69813b463223e9eed2/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}