Decentralized Coordination Under Partial Observability

1,511 concepts1,149 papers31 proposals

In multi-agent systems, the ideal scenario is full observability: every agent knows the exact state of the world, the positions of its teammates, and the actions everyone plans to take. Under full observability, coordination is a straightforward optimization problem.

Real-world deployments, however, rarely afford this luxury. Robots navigating a disaster zone, sensors distributed across a vast field, or autonomous vehicles at a busy intersection must operate under partial observability. They see only what is within their immediate sensory range. This creates a fundamental information gap: how can independent agents execute a cohesive, globally optimal strategy when they only possess local, incomplete pieces of the puzzle?

To bridge this gap, modern multi-agent research focuses on mechanisms that allow agents to reconstruct global context, anticipate teammate intentions, and establish common grounding without relying on constant, unrestricted communication.

Centralized Training with Decentralized Execution

The foundational paradigm for tackling partial observability is Centralized Training with Decentralized Execution (CTDE). The core intuition behind CTDE is simple: during training—such as in a simulator—we can "cheat" by giving the learning algorithm access to the global state. However, the policies we train must remain decentralized, relying strictly on local observations during deployment.

Training (Centralized Critic / Joint Value Function)
  [ Global State + All Local Observations ]
                     │
                     ▼
Execution (Decentralized Actors / Individual Policies)
  [ Agent 1 Obs ]   [ Agent 2 Obs ]   [ Agent 3 Obs ]

A primary challenge in this setup is the credit assignment problem. If a team of agents receives a single global reward (e.g., winning a game), how does an individual agent operating with a limited field of view know whether its specific local action contributed to that success?

To solve this, Counterfactual Multi-Agent (COMA) Policy Gradients, developed by Jakob Nicolaus Foerster et al., uses a centralized critic to compute a counterfactual baseline. The critic computes the expected reward if a specific agent's action were replaced with a default or alternative action, while keeping all other agents' actions fixed. This isolates an individual agent's marginal contribution, allowing it to learn effective local policies from a global reward signal.

Another prominent approach to CTDE is value function factorization. In QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning, Tabish Rashid et al. enforce a structural monotonicity constraint on how individual agent utilities are mixed into a joint action-value function ($Q_{tot}$):

$$\frac{\partial Q_{tot}}{\partial Q_a} \ge 0 \quad \forall a$$

This mathematical constraint ensures that a maximization operation performed locally by an individual agent on its own utility ($Q_a$) aligns perfectly with maximizing the global joint action-value function ($Q_{tot}$). Consequently, agents can execute optimal decentralized choices based solely on local observations, confident that their independent choices sum up to a coordinated global strategy.

Reconstructing the Hidden State

While CTDE leverages global information during training, some environments hide the global state entirely, or the deployment conditions differ so drastically from training that agents must actively infer the missing context on the fly.

When the global environment state is completely hidden, agents can treat their stream of local observations as clues to reconstruct a latent representation of the world. For instance, Zhiwei Xu et al. introduced SIDE: I Infer the State I Want to Learn, a framework that uses variational inference and graph neural networks to synthesize local views into a coherent latent global state. By learning to predict the underlying state transitions from historical local data, agents effectively generate their own internal "cheat codes" to guide coordination.

However, physical state reconstruction is only half the battle; agents must also achieve teammate awareness. Under partial observability, an agent cannot see what its peers are experiencing or planning. To overcome this information bottleneck, Jiahan Cao et al. proposed LINDA: multi-agent local information decomposition for awareness of teammates. Instead of relying on standard neural networks to implicitly extract features, LINDA explicitly isolates teammate awareness as a distinct learned representation. This allows an agent to explicitly predict the states and actions of its peers, transforming raw, localized data into a predictive model of team behavior.

Perspective Taking and Common Grounding

When explicit communication is costly, bandwidth-limited, or absent, coordination requires agents to reason about perspectives. This concept is deeply tied to pragmatics—the study of how context enriches meaning.

In collaborative dialogue and situated tasks, partial observability and agent symmetry act as natural catalysts for social reasoning. As surveyed by Daniel Fried et al. in Pragmatics in Language Grounding: Phenomena, Tasks, and Modeling Approaches, agents cannot rely on literal interpretations alone when their views of the shared world only partially overlap. Instead, they must proactively model the boundaries of their partner's hidden perspective to resolve ambiguities.

To operationalize this, Daniel Fried et al. developed Reference-Centric Models for Grounded Collaborative Dialogue. The model incorporates a pragmatic generation strategy that simulates how a partner would interpret a potential utterance given their restricted view. By running this internal mental simulation, the agent selects actions or language that maximize the probability of successful common grounding, explicitly identifying which entities are mutually observable and which are unique to its own perspective.

Agent A (Local View) ──► Simulates Partner's Interpretation ──► Pragmatic Action
                               ▲
Agent B (Hidden View) ─────────┘

Where explicit language is unavailable, agents can establish coordination through mathematical or structural alignment. Christian Schroeder de Witt et al. introduced Multi-Agent Common Knowledge Reinforcement Learning (MACKRL), which leverages common knowledge hierarchies to bridge independent and centralized learning. By acting only on information that is mutually known to be known, agents avoid catastrophic miscoordination.

Alternatively, agents can bypass complex perspective-taking altogether by correlating their policies through external signals. In Signal Instructed Coordination in Cooperative Multi-agent Reinforcement Learning, Liheng Chen et al. demonstrated that broadcasting a shared random signal to all agents—combined with an information-theoretic loss—allows decentralized agents to correlate their actions. This mathematical correlation expands the decentralized joint policy space to match the expressivity of a fully centralized controller, even under strict partial observability constraints.

Coordinating in Open and Ad Hoc Teams

The challenge of partial observability peaks in ad hoc teamwork, where an agent must collaborate with unfamiliar teammates on the fly without pre-established communication protocols or shared training phases.

In these open environments, an agent cannot assume it knows the policies or capabilities of its peers. To address this, Arrasy Rahman developed Graph-based Policy Learning (GPL), a framework that uses graph neural networks to dynamically model joint action values. GPL allows an agent to disentangle its individual contributions from those of an arbitrary, changing number of teammates, maintaining robust coordination despite incomplete information about partner strategies.

To handle rapid behavioral shifts in unknown partners, Pengjie Gu et al. proposed Online Ad Hoc Teamwork under Partial Observability (ODITS). Rather than categorizing teammate strategies into rigid, predefined profiles, ODITS models the fluid "teamwork situation" as a continuous latent variable. By continuously updating this latent variable from local observations, the agent rapidly adapts its policy to complement the real-time behavior of its partners.

On a broader scale, this capacity for distributed estimation mirrors fundamental limits in sensor networks. As José M. F. Moura proved in Convergence Rate Analysis of Distributed Gossip (Linear Parameter) Estimation: Fundamental Limits and Tradeoffs, a network of sparsely connected nodes can achieve the same estimation accuracy as a fully centralized system through localized, peer-to-peer information sharing (gossip algorithms).

Similarly, in human-robot teaming, Esmaeil Seraj introduced Mixed-Initiative Multiagent Apprenticeship Learning for Human Training of Robot Teams (MixTURE). MixTURE allows a human to guide a robot team by demonstrating physical actions without managing their communication. The underlying AI automatically evolves an emergent communication protocol via mutual-information maximization, enabling the robots to resolve their local information gaps autonomously.

By combining centralized training structures, latent state inference, pragmatic perspective modeling, and ad hoc adaptation, modern multi-agent systems successfully navigate the constraints of partial observability. These mechanisms ensure that even when agents see only a fraction of the world, they can still act effectively as a unified whole.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 1114 papers
  2. Version 1 · 2026-10-04 · 1437 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_partial_observability_partial_observabil_7c3ebb6d,
  title = {partial observability (partial observability in multi-agent systems)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/partial-observability-partial-observability-in-multi-agent-systems/rdr_7c3ebb6d6ce0437a973d17d5a2a3fa9d},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 2, frozen text):

@misc{lacuna_partial_observability_partial_observabil_7c3ebb6d_v2,
  title = {partial observability (partial observability in multi-agent systems)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/partial-observability-partial-observability-in-multi-agent-systems/rdr_7c3ebb6d6ce0437a973d17d5a2a3fa9d/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake