Abstract
In this paper, we discuss the learning of generalised policies for probabilistic and classical planning problems using Action Schema Networks (ASNets). The ASNet is a neural network architecture that exploits the relational structure of (P)PDDL planning problems to learn a common set of weights that can be applied to any problem in a domain. By mimicking the actions chosen by a traditional, non-learning planner on a handful of small problems in a domain, ASNets are able to learn a generalised reactive policy that can quickly solve much larger instances from the domain. This work extends the ASNet architecture to make it more expressive, while still remaining invariant to a range of symmetries that exist in PPDDL problems. We also present a thorough experimental evaluation of ASNets, including a comparison with heuristic search planners on seven probabilistic and deterministic domains, an extended evaluation on over 18,000 Blocksworld instances, and an ablation study. Finally, we show that sparsity-inducing regularisation can produce ASNets that are compact enough for humans to understand, yielding insights into how the structure of ASNets allows them to generalise across a domain.
Introduction
In traditional automated planning, an agent typically solves a single problem instance from scratch. A standard planner lacks the inherent ability to "learn" that the general strategy for a domain, such as Blocksworld, remains the same regardless of the number of objects involved. Deep learning offers a path toward learning these generalized strategies, but standard neural networks require fixed-size inputs and outputs. In planning, the number of objects—and thus the number of possible actions and propositions—changes between instances.
Furthermore, deep networks often struggle with "long-range" dependencies in large state spaces, a challenge known as the receptive field limitation. If a network is only a few layers deep, it can only "see" a limited number of steps ahead in a chain of related actions, which is often insufficient for complex, large-scale problems.
Methods: Action Schema Networks
The authors introduce Action Schema Networks (ASNets), a specialized architecture where the network structure is derived directly from the planning domain's relational logic.
Relational Weight Sharing
The core innovation of ASNets is that weights are tied to action schemas and predicates rather than specific objects. For example, the logic for a drive action is identical whether an agent is in a small town or a large city. This relational weight sharing allows a single set of weights to be applied to any problem instance in a domain, regardless of its size.
Architecture and Information Flow
An ASNet alternates between layers of action modules and proposition modules. These modules pass messages to their neighbors based on the domain's preconditions and effects, effectively performing a form of relational convolution.
To overcome the receptive field limitation, the network incorporates features from traditional domain-independent heuristics (like LM-cut landmarks). These act as "global eyes," providing the local network structure with information about distant goals. The network is trained using a teacher-student approach (similar to DAgger), where it learns to mimic the optimal actions of a traditional planner on small training instances.
Experimental Evaluation
ASNets were evaluated on seven diverse domains, including probabilistic (e.g., Triangle Tireworld, Exploding Blocksworld) and deterministic (e.g., Gold Miner) versions. A massive test set of 18,300 Blocksworld instances ranging from 18 to 50 blocks was used for extended evaluation.
Key Results
- Generalisation: ASNets trained on problems with only 5–9 blocks solved problems with 50 blocks with near-perfect success rates, demonstrating a powerful inductive bias.
- Superiority over Baselines: In domains like CosaNostra Pizza and Triangle Tireworld, ASNets achieved 100% coverage, while state-of-the-art heuristic planners (like LRTDP and LAMA) struggled as problem size increased.
- Speed and Interpretability: Once trained, an ASNet produces actions almost instantaneously. Furthermore, by using $L_1$ (sparsity-inducing) regularization, the authors produced compact networks that could be written as simple equations, allowing humans to verify the AI's decision-making logic.
Conclusion
ASNets represent a significant advancement in generalized planning by combining the strengths of symbolic heuristics with the learning capabilities of deep relational networks. While current limitations include the overhead of grounding every possible action and a lack of support for complex PDDL features like numeric variables, the architecture successfully learns the underlying "logic" of a domain rather than just a path to a specific goal.
Summary updated