ASNets: Deep Learning for Generalised Planning

Sam Toyer, Sylvie Thiébaux, Felipe W. Trevizan, Lexing Xie

Abstract

In this paper, we discuss the learning of generalised policies for probabilistic and classical planning problems using Action Schema Networks (ASNets). The ASNet is a neural network architecture that exploits the relational structure of (P)PDDL planning problems to learn a common set of weights that can be applied to any problem in a domain. By mimicking the actions chosen by a traditional, non-learning planner on a handful of small problems in a domain, ASNets are able to learn a generalised reactive policy that can quickly solve much larger instances from the domain. This work extends the ASNet architecture to make it more expressive, while still remaining invariant to a range of symmetries that exist in PPDDL problems. We also present a thorough experimental evaluation of ASNets, including a comparison with heuristic search planners on seven probabilistic and deterministic domains, an extended evaluation on over 18,000 Blocksworld instances, and an ablation study. Finally, we show that sparsity-inducing regularisation can produce ASNets that are compact enough for humans to understand, yielding insights into how the structure of ASNets allows them to generalise across a domain.

Introduction

Automated planning typically involves finding a sequence of actions to transition from an initial state to a goal state. While traditional planners are effective at solving specific problem instances, they often lack the ability to transfer knowledge across different instances of the same domain. For example, a planner that solves a 5-block Blocksworld problem does not inherently gain insights that facilitate solving a 50-block problem.

Applying deep learning to this domain presents significant challenges. Standard neural networks require fixed-size inputs and outputs, whereas planning problems vary in size (number of objects, actions, and propositions). Furthermore, planning problems are inherently relational, defined by predicates like on(A, B). Standard architectures like CNNs or RNNs are not designed to exploit this relational structure. Action Schema Networks (ASNets) address these issues by providing a size-invariant architecture that understands the underlying action schemas of a planning domain.

Methods

The Action Schema Network (ASNet) is a deep neural network whose architecture is automatically generated from the PPDDL (Planning Domain Definition Language) description of a planning domain.

Relational Weight Sharing and Architecture

The core of the ASNet consists of alternating "action layers" and "proposition layers." Each layer contains modules corresponding to the ground actions and propositions of the specific planning instance. Crucially, ASNets employ relational weight sharing: all action modules instantiated from the same action schema (e.g., pick-up(x)) share the same weights, regardless of the specific object x. Similarly, proposition modules for the same predicate share weights. This allows the network to learn general logic applicable to any problem instance in the domain.

The architecture of an Action Schema Network (ASNet), showing the alternating layers of action and proposition modules.

Connectivity and Receptive Fields

The network's connectivity mirrors the relational structure of the planning problem. An action module is connected to the proposition modules representing its preconditions and effects. Conversely, a proposition module is connected to the action modules that can affect it. By stacking multiple layers, the network increases its "receptive field," allowing it to consider the long-range consequences of actions. To further assist with long-range planning, the authors incorporate heuristic inputs (such as LM-cut landmarks) and action counts as global features, acting as "binoculars" for the network.

Training via Imitation

ASNets are trained using a teacher-student paradigm. A traditional planner serves as the teacher, providing optimal or near-optimal solutions for small problem instances. The ASNet learns to mimic these decisions using an imitation learning regime similar to the DAgger algorithm, which helps the network remain robust even when it deviates from the teacher's path.

Experimental Evaluation

The authors evaluated ASNets on seven diverse domains, including probabilistic (CosaNostra Pizza, Exploding Blocksworld, Triangle Tireworld) and deterministic (Gold Miner, Matching Blocksworld) settings.

Key Results

  • Zero-Shot Scaling: ASNets trained on small instances (e.g., 5-9 blocks) were able to solve significantly larger instances (up to 50 blocks) almost instantaneously.
  • Superior Coverage: In challenging domains like Triangle Tireworld, ASNets maintained high success rates as problem size increased, whereas traditional state-of-the-art planners like LRTDP or LAMA saw performance degradation.
  • Blocksworld Mastery: In an extensive evaluation involving over 18,000 Blocksworld instances, the ASNet achieved a 100% success rate, demonstrating that it had successfully learned the domain's underlying rules rather than just memorizing paths.
  • Interpretability through Sparsity: By applying $L_1$ regularisation, the authors generated sparse networks. These compact networks are human-interpretable, often reducing to a few simple logical rules that explain the learned policy, turning the "black box" of deep learning into a "white box."

Conclusion

ASNets represent a significant step forward in generalised planning by combining the strengths of deep learning with the relational structure of classical planning. By exploiting action schemas and weight sharing, they achieve remarkable zero-shot generalisation. While current limitations include handling high-arity schemas and the requirement for grounding, future work on "lifted" architectures may further expand the applicability of this approach to even more complex, real-world planning scenarios.

Summary updated

Abstract

In this paper, we discuss the learning of generalised policies for probabilistic and classical planning problems using Action Schema Networks (ASNets). The ASNet is a neural network architecture that exploits the relational structure of (P)PDDL planning problems to learn a common set of weights that can be applied to any problem in a domain. By mimicking the actions chosen by a traditional, non-learning planner on a handful of small problems in a domain, ASNets are able to learn a generalised reactive policy that can quickly solve much larger instances from the domain. This work extends the ASNet architecture to make it more expressive, while still remaining invariant to a range of symmetries that exist in PPDDL problems. We also present a thorough experimental evaluation of ASNets, including a comparison with heuristic search planners on seven probabilistic and deterministic domains, an extended evaluation on over 18,000 Blocksworld instances, and an ablation study. Finally, we show that sparsity-inducing regularisation can produce ASNets that are compact enough for humans to understand, yielding insights into how the structure of ASNets allows them to generalise across a domain.
BibTeX
@article{toyer2020asnets,
  author = {Sam Toyer and Sylvie Thiébaux and Felipe W. Trevizan and Lexing Xie},
  title = {{ASNets: Deep Learning for Generalised Planning}},
  journal = {J. Artif. Intell. Res.},
  year = {2020},
  url = {https://openreview.net/forum?id=AjMIVWpPXSs},
}
Figure