Planning under Distribution Shifts with Causal POMDPs

Karthika Mohan, Matteo Ceriscioli

TL;DR

This paper introduces a novel framework for planning in partially observable environments that experience distribution shifts (changes in environment dynamics). By leveraging causal models and representing these shifts as interventions, the authors ensure that planning remains computationally tractable, even when the exact nature of the shift is unknown.

What Problem Does This Paper Address?

In the real world, intelligent agents often operate in environments that are not static. The underlying dynamics of the environment, or the way states transition, can change over time. This phenomenon, known as a "distribution shift" or "domain shift," can render previously learned plans and policies ineffective. Imagine a robot trained to navigate a sunny, flat urban landscape. If deployed in a snowy, mountainous region, its learned model of friction, visibility, and even its own mechanical limitations might become inaccurate, leading to failure.

Traditional planning methods, particularly for Partially Observable Markov Decision Processes (POMDPs) where the agent doesn't have complete information about the environment's state, rely on a fixed model of the environment. When this model becomes outdated due to distribution shifts, these methods break down. The challenge is to develop planning strategies that can adapt to these shifts, identify what has changed, and continue to make optimal decisions, all while dealing with the inherent partial observability.

Methods

The core of this paper's contribution lies in its use of causal POMDPs. Instead of treating the environment as a black box, they model it using a causal structure. This allows them to represent distribution shifts not as arbitrary changes, but as deliberate interventions on specific components of the causal model.

Here's a breakdown of their approach:

  • Causal Factorization of POMDPs: They represent the state of a POMDP as a set of state variables and use a causal influence diagram (CID) to model the dependencies between these variables. This "factored" representation is key.
  • Modeling Shifts as Interventions: Distribution shifts are framed as "stochastic shift interventions" on this causal model. This means they can mathematically describe how a change in one part of the causal graph affects the overall probability distributions.
  • Joint Belief over State and Domain: To handle unknown shifts, the agent needs to maintain not just a belief about the current state of the environment, but also a belief about which "domain" (i.e., which specific shift) it's currently in. They develop a Bayesian update rule for this joint belief.
  • Preserving Tractability: A crucial aspect of POMDP planning is the piecewise linear and convex (PWLC) property of the value function. This property allows for efficient planning algorithms. The authors prove that their causal framework, even with unknown distribution shifts, preserves this PWLC structure. This is a significant theoretical result, as it means existing, efficient POMDP planning algorithms can still be applied.

Datasets

Not discussed in the paper.

Key Insights and Ideas

  • Causality as a Bridge for Robustness: The paper elegantly demonstrates how thinking causally about the environment can provide a principled way to handle distribution shifts. By understanding why things change (as interventions on causal mechanisms), agents can reason more effectively about the consequences.
  • Interventions as a Unified Framework: Representing both environment dynamics and distribution shifts as interventions provides a consistent mathematical language for reasoning about these complex scenarios.
  • The Power of Joint Belief: The insight that an agent needs to maintain a belief about both the state and the potential domain shifts is critical for planning under uncertainty about the environment's evolution.
  • PWLC Preservation is Key: The theoretical guarantee that the value function remains PWLC is a major contribution. It means that despite the added complexity of unknown distribution shifts, the problem doesn't become computationally intractable, allowing for practical application of existing planning techniques.

Key Results

The paper's primary contribution is theoretical:

  • A formal framework for planning under distribution shifts using causal POMDPs.
  • A method to represent distribution shifts as interventions on causal models.
  • A Bayesian update rule for maintaining a joint belief over states and domains.
  • A proof that the value function of a causal POMDP under unknown distribution shifts remains piecewise linear and convex (PWLC). This is crucial for maintaining planning tractability.

The authors do not present empirical results or quantitative performance metrics on specific datasets, as the paper focuses on establishing the theoretical foundation.

Limitations and Future Work

  • Theoretical Focus: The paper lays a strong theoretical groundwork. A natural next step would be to demonstrate the practical efficacy of this framework through empirical evaluations on benchmark problems or real-world robotics tasks.
  • Complexity of Interventions: While interventions are a powerful modeling tool, defining and learning the parameters of stochastic shift interventions can still be challenging in practice.
  • Scalability: The paper focuses on the theoretical tractability of planning. Investigating the practical scalability of these methods to larger state spaces and more complex causal structures would be valuable.
  • Detection and Identification: While the framework enables reasoning about shifts, explicitly developing algorithms for detecting and identifying the specific shift in real-time could be a significant area for future work.

Summary updated

Abstract

In the real world, planning is often challenged by distribution shifts. A model of the environment obtained under some circumstances may not remain valid as the distribution of states and the environment dynamics change, causing learned strategies to fail. A distribution shift that alters the environment or its dynamics is known as a domain shift. In this work, we propose a theoretical framework for planning under partial observability, using Partially Observable Markov Decision Processes (POMDPs) equipped with a causal factorization. By representing domain shifts as interventions on the causal model induced by the causal factorization, this framework enables evaluating plans under hypothesized shifts and actively identifying which components of the environment have changed. We show how to maintain and update a belief about the state and the domain and demonstrate that the value function remains piecewise linear and convex in this augmented belief space.
BibTeX
@inproceedings{mohan2026planning,
  author = {Karthika Mohan and Matteo Ceriscioli},
  title = {{Planning under Distribution Shifts with Causal POMDPs}},
  booktitle = {ICAPS},
  year = {2026},
  url = {https://openreview.net/forum?id=f3HOjFonDl},
}
Figure