Probabilistic Programs as an Action Description Language

Ronen I. Brafman, David Tolpin, Or WertheimAAAI 2023 · DOI

Introduction

In the field of autonomous agents, the ability to model the environment—defining the actions available and their potential consequences—is fundamental to effective planning. Historically, this modeling has been the domain of specialized Action Description Languages (ADLs) such as STRIPS, PDDL, and RDDL. While these languages have served the AI planning community for decades, they often present a significant "usability gap" for modern software engineers.

This paper argues for a paradigm shift: replacing niche ADLs with Probabilistic Programming Languages (PPLs) like Pyro, Stan, or Church. By adopting PPLs, researchers can leverage familiar syntax and powerful inference tools to model complex, stochastic, and "open-world" environments that remain out of reach for traditional frameworks.

The Limitations of Traditional ADLs

Traditional planning languages are rooted in formal logic, which, while mathematically rigorous, lacks the flexibility of general-purpose programming. Two primary issues hinder their adoption and efficacy:

  1. The Usability Gap: The syntax of languages like PDDL is often foreign to developers. More importantly, these languages lack basic constructs that programmers take for granted, such as loops, recursion, and complex data structures (e.g., tensors or trees).
  2. The Closed-World Assumption: Most traditional ADLs assume a static set of objects. In real-world scenarios—such as a robot cleaning a room where it may discover new items or break existing ones—modeling a dynamic environment becomes an administrative nightmare in a closed-world framework.

PPLs as a Modern Alternative

The authors propose using PPLs as the primary medium for describing planning domains. Unlike a standard simulator, which acts as a "black box," a PPL represents a statistical model as an executable program. This approach provides several distinct advantages:

  • Generative Modeling: PPLs are designed to describe how states evolve stochastically, making them ideal for uncertain environments.
  • Built-in Inference: Sophisticated algorithms like Markov Chain Monte Carlo (MCMC) and Sequential Monte Carlo (SMC) are native to PPLs, allowing for robust execution monitoring and plan validation.
  • Automatic Differentiation: Modern PPLs support "auto-diff," enabling planners to utilize gradient-based optimization to refine policies and actions.
  • Expressive Control Flow: Standard constructs like for loops and if-then-else statements allow for more compact and readable code compared to the expanded graph structures required by languages like RDDL.

Illustrative Domains and Results

To demonstrate the superiority of PPLs, the paper examines three challenge domains where traditional ADLs struggle:

  1. Locusts Swarm: This domain involves multi-stage events where a swarm moves and consumes resources. In a PPL, this is represented by a simple loop. In RDDL, the same logic requires manually adding layers to a graph, causing the code size to grow linearly with the complexity of the event. The PPL implementation remains a fixed size regardless of the swarm's duration.
  2. Open-World Room Cleaning: This scenario features a robot that discovers new objects over time. PPLs handle this dynamic object creation naturally, whereas traditional ADLs like RDDL or PDDL cannot easily capture a world where the state space's dimensionality changes.
  3. Sailing Domain: A classic benchmark used to show how PPLs can optimize policy parameters through automatic differentiation, streamlining the path from model description to optimal behavior.

Key Insights: A Programmer-First Philosophy

The core argument is that planning technology must align with the tools and mental models of software engineers to achieve widespread adoption. A PPL looks like Python or C++, not a logic textbook. Furthermore, PPLs treat inference as a "first-class citizen." An agent can query its own model—asking, "Given my current observations, how likely is it that my plan is still on track?"—which is essential for robust execution in the real world.

Challenges and Future Work

Despite their expressivity, PPLs introduce new challenges. The "Black Box" problem arises because arbitrary code is harder for symbolic reasoners to analyze than restricted logic formulas. Additionally, the high expressivity of PPLs can lead to computational complexity, with some inference tasks becoming technically undecidable.

Future research should focus on:

  • Developing planning algorithms that exploit the specific structure of PPL code (e.g., identifying deterministic sub-routines).
  • Creating compilers to translate PPL models into restricted formats for use with existing high-speed planners.
  • Learning action models directly as PPL programs from raw observational data.

Conclusion

By transitioning from rigid Action Description Languages to flexible Probabilistic Programs, the AI planning community can bridge the gap between theoretical modeling and practical software engineering. This shift enables the creation of more robust, scalable, and "open-world" agents capable of navigating the complexities of the real world.

Summary updated

BibTeX
@inproceedings{brafman2023probabilistic,
  author = {Ronen I. Brafman and David Tolpin and Or Wertheim},
  title = {{Probabilistic Programs as an Action Description Language}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2023},
  doi = {10.1609/aaai.v37i13.26790},
}
Figure