Abstract: We extend Action Schema Networks (ASNets) to learn generalised policies for numeric planning, which features quantitative numeric state variables, preconditions and effects. We propose a neural network architecture that can reason about the numeric variables both directly and in context of other variables. We also develop a dynamic exploration algorithm for more efficient training, by better balancing the exploration versus learning tradeoff to account for the greater computational demand of numeric teacher planners. Experimentally, we find that the learned generalised policies are capable of outperforming traditional numeric planners on some domains, and the dynamic exploration algorithm to be on average much faster at learning effective generalised policies than the original ASNets training algorithm.
Introduction
Generalized planning is often considered the "holy grail" of automated planning: the goal is to learn a single strategy (a policy) that can solve any problem instance within a specific domain, regardless of its size or complexity. While significant progress has been made in classical (propositional) planning, numeric planning—which involves continuous or integer variables like distance, fuel, or capacity—has remained a challenge for deep learning approaches.
Existing neural architectures often struggle with numeric data due to scale sensitivity (where a policy trained on small values fails on larger ones), the high computational cost of numeric "teacher" planners used for imitation learning, and the need to respect the relational structure of the problem. This paper, presented at ICAPS 2024, introduces Numeric Action Schema Networks ($\nu$-ASNets) to overcome these hurdles.
The $\nu$-ASNets Architecture
The $\nu$-ASNets architecture builds upon the original Action Schema Network (ASNet) framework, which uses a graph-like structure where nodes represent actions and propositions. To handle numeric variables (fluents), the authors introduce two critical components:
- Fluent Modules: These allow the network to reason directly about quantitative values, such as current fuel levels or object weights.
- Comparison Modules: Instead of relying solely on raw numbers, these modules evaluate the relationships between fluents (e.g., checking if
weight(o) + load(r) ≤ limit(r)). This is a key driver for generalization, as it allows the network to recognize logical constraints that remain invariant even when absolute values change. - Numeric Landmarks: The input features are augmented with hybrid landmarks derived from the problem's structure, helping the network understand long-term goals that might otherwise fall outside its immediate receptive field.
Efficient Training via Dynamic Exploration
In numeric planning, the "teacher" (the expert planner providing training labels) is often the most significant bottleneck. Planners like ENHSP are computationally expensive, making standard imitation learning loops prohibitively slow.
To address this, the authors developed a Dynamic Exploration algorithm. Unlike the original ASNet trainer, which explored every state in a sampled trajectory, this version intelligently prunes "stale" or redundant states and balances exploration time against learning time. By focusing on guiding the network back to valid trajectories rather than exhaustive exploration, the algorithm makes training on complex numeric domains feasible.
Experimental Evaluation
The researchers evaluated $\nu$-ASNets on a representative set of benchmarks from the International Planning Competition (IPC) 2023 Numeric Track, including logistics tasks (Drone, Rover, TPP) and resource management problems (Zenotravel). Models were trained on very small instances (e.g., 3 to 6 objects) and tested on much larger, unseen problems.
Key results include:
- Outperforming Traditional Planners: $\nu$-ASNets achieved higher coverage (solved more problems) than the state-of-the-art ENHSP planner on several domains, including Delivery, FO-Counters, and TPP.
- Massive Generalization: In the Delivery domain, the model was trained on instances with only 10 items but successfully solved test instances with up to 42 items.
- Training Efficiency: The new Dynamic Exploration algorithm was significantly faster—often by an order of magnitude—than the original ASNet training method, without sacrificing policy quality.
Conclusion and Future Work
$\nu$-ASNets represent a significant leap for generalized planning by bringing numeric reasoning into the fold of deep learning. The authors found that reasoning about the interaction between fluents via comparison modules is often more important for generalization than the raw values themselves.
While highly effective, the current architecture still faces challenges with plans requiring highly repetitive actions and does not yet explicitly handle universal or existential quantifiers. Future work will focus on architectures capable of predicting not just which action to take, but how many times to apply it, further bridging the gap between neural policies and complex numeric reasoning.
Summary updated