Teacher-Proposed Feature Pools for Learning General Planning Policies

A general planning policy is a small set of rules that solves every instance of a domain, whether it has five blocks or fifty. The best current way to learn one, the D2L method of Learning General Planning Policies from Small Examples Without Supervision, works in three steps. It enumerates every candidate feature that a formal grammar can build from the domain's predicates, for example "number of blocks not on their target block." It then encodes the policy requirements as a weighted satisfiability problem: goal states must be told apart from non-goal states, good actions must be selectable, and a distance-like quantity must decrease so the policy cannot loop. Finally, a solver picks the cheapest subset of features that satisfies those requirements. Policies learned this way on 4–5 objects solve instances with 30 or more. The method has a known ceiling. The candidate pool grows combinatorially with feature complexity, and every candidate must be evaluated on every state of the training instances, so the solver's problem becomes too large exactly when a domain needs complex features or larger training instances.

We propose using a large language model as a teacher that proposes the candidate feature vocabulary and assigns prior weights to it. The satisfiability solver stays in its role as the student-side verifier that picks and certifies the final policy. This is a form of knowledge distillation in which the teacher's knowledge ends up in a handful of checkable symbolic features rather than in a neural student. The central claim is that a teacher-proposed pool, merged with a cheap low-complexity enumeration, lets the solver find sound policies that pure enumeration either finds only at much greater solver cost or cannot reach within a fixed budget. It should also do this without giving up the correctness guarantees that make D2L policies trustworthy.

Core Claim and What Is New

The closest existing work is Declarative Knowledge Distillation from Large Language Models for Visual Question Answering Datasets. It prompts GPT-4 to write whole Answer Set Programming rules (logic programs run by a solver) for visual question answering, and repairs them through a loop of syntax and error feedback. It reports 95–100% recovery of deliberately deleted operations on GQA. That paper motivates our proposal because it shows language models can propose correct relational logic. It also constrains our design in two ways. First, its authors found that GPT-3.5 struggled with strict logical syntax. Second, the rule search is heuristic and comes with no minimality or soundness guarantee; the authors themselves name integration with inductive logic programming as open work.

We differ in what the teacher is asked to produce. The teacher does not write the final program. It writes candidate features: short, typed description-logic expressions and a few extended forms. Correctness is then decided entirely by the combinatorial selection of D2L and of the sketch learner in Learning Sketches for Decomposing Planning Problems into Subproblems of Bounded Width. A sketch, in that work, is a set of rules that split a hard planning problem into easy subproblems. Any teacher hallucination is harmless in this setup: a wrong feature is simply never selected. The teacher's knowledge therefore affects which policy is found and how cheaply, but never whether that policy is correct.

We also transfer the teacher's graded confidence, not only its discrete proposals. This follows the spirit of soft-label distillation in Transferring knowledge from a RNN to a DNN, where a student trained on a teacher's probability distributions rather than hard labels reduced word error rate from 4.5% to 3.9%. Here the soft signal enters the solver's objective. D2L weights each feature by its syntactic complexity. We instead minimize

$$\sum_{f \in S} \big(\lambda\, c(f) - \mu \log p_T(f)\big)$$

over selected feature sets $S$. Here $c(f)$ is the grammar complexity of feature $f$, and $p_T(f)$ is the teacher's normalized probability that $f$ is relevant. We estimate $p_T(f)$ from how often $f$ appears across $k$ sampled proposal rounds. The constants $\lambda, \mu \ge 0$ trade off simplicity against the teacher's prior. Setting $\mu = 0$ recovers D2L's objective exactly on the same pool.

Research Program

The system has three stages. In the first, the teacher is given the PDDL domain file (the standard planning language that declares predicates and actions), a few small training instances, and a short specification of the feature language. It returns several hundred candidate features. Each proposal is parsed, rejected if it is ill-formed, and removed if it has the same values on all training states as a feature already in the pool; D2L already applies this last filter to its own pool. Syntax errors go back to the teacher once, borrowing the repair loop of the visual question answering work.

The feature language starts as D2L's description-logic grammar. We then add two constructs that the grammar either lacks or only reaches at high complexity: counting over role chains, and transitive closure of a relation. This addresses the expressivity limit noted in the research file. A grammar can only find features it can express, and raising the complexity bound far enough to express these features makes blind enumeration explode.

In the second stage, the teacher's pool is merged with a low-complexity enumerated pool. This keeps the method from ever doing worse than enumeration at low complexity. The merged pool is then evaluated on the expanded state graphs of the training instances.

In the third stage, the resulting weighted satisfiability problem, or the answer-set problem in the sketch version, is solved. The solution is tested on much larger instances.

We state one limit up front. The teacher shrinks the number of features, not the state-space expansion itself. The solver's problem size scales roughly with the number of features times the number of transitions. A pool that is ten times smaller therefore buys either a proportional reduction in solver effort or room for training instances with about ten times more transitions. Instances that cannot be expanded at all remain out of reach. We treat this as a scope boundary rather than claiming to remove the grounding bottleneck.

We will run open-weight teachers at two sizes. This tests whether the benefit requires a frontier model, given the GPT-3.5 failure reported in the visual question answering paper, and it keeps the study reproducible.

Baselines

The do-nothing reference is D2L run exactly as published, using its grammar, its complexity-weighted objective, and its smallest training instances. The published results set the bar. D2L finds a Gripper policy in about 2 seconds, and policies trained on 4–5 objects solve test instances with 30 or more objects (Learning General Planning Policies from Small Examples Without Supervision). For the sketch variant, the strongest existing result is width-1 sketches reaching 100% coverage (30/30) on every tested domain, including Spanner, where the standard planner LAMA fails (Learning Sketches for Decomposing Planning Problems into Subproblems of Bounded Width). The file reports no per-domain solver time or pool size for domains where D2L times out, so we will measure those ourselves.

The strongest competitor is D2L with a larger complexity bound, given the same wall-clock and memory budget as our full pipeline including teacher inference time. This is the fair test of whether the teacher adds anything beyond extra compute. Beating it means one of two things. Either we find sound policies on domains or training sizes where the scaled-up enumeration times out, or we match its coverage at an order of magnitude less solver time.

The key ablation is a random subset of the enumerated pool, matched in size to the teacher's pool and also merged with the low-complexity pool. This isolates whether the teacher's choices carry information beyond simply having fewer features. A second ablation sets $\mu = 0$, which isolates the contribution of the soft prior from that of the proposals.

Finally, we run a direct-generation baseline in the style of the visual question answering work. The teacher writes the full policy rules, and a verifier checks them on the training states and repairs them through the same error-feedback loop. It gets the same number of teacher calls as our method.

Why not just…?

Why not just raise D2L's complexity bound and use a faster solver? This is the most serious threat. D2L already removes features with duplicate values on the training states, so the growth in candidates may be milder in practice than the grammar suggests. The equal-budget baseline above settles the question directly. If scaled-up enumeration matches us on every domain within budget, the claim fails for pool construction, and what remains is only the expressivity result from the counting and closure constructs.

Why not let the language model write the policy directly and skip the solver? The visual question answering work shows these rules are often almost right but need repair, and even after repair nothing guarantees they terminate on larger instances. Our direct-generation baseline measures this. We expect it to pass the training states but loop or dead-end on larger test instances more often than solver-selected policies, which are required by construction to make a distance-like quantity decrease. If it generalizes equally well, the solver's role shrinks to certification, and we will report that outcome.

Why not train a neural policy distilled from the teacher? Such a policy would lose the inspection and generalization properties that motivate this line of work. D2L's 1–5-rule policies are fully readable and extrapolate from 5 objects to 30. The research file contains no reported result on Lacuna for neural policies on these domains, so we exclude this baseline from the claims rather than asserting it fails.

Is the teacher just recalling published features? Planning domains and D2L's features appear in pretraining text, so this is a real risk. We will rerun every experiment with predicate and object names replaced by meaningless symbols. We will also include domains whose names are disguised or that the teacher is unlikely to have seen in feature form. A large drop under renaming would mean the teacher is recalling features rather than reasoning about the domain, and we would report the renamed numbers as the main result.

Proposed Experiments

The domains are the ones used by D2L and the sketch learner: Gripper, Blocksworld (both the clear-a-block and on-a-block goal variants), Spanner, Miconic, Visitall, and Childsnack. We train on the published tiny instances and on a ladder of progressively larger ones. We test on instances with 30 to 100 objects, following both papers.

The measurements are:

  • whether a sound policy or width-1 sketch is found within budget;
  • solver time and peak size of the satisfiability problem;
  • the largest training instance that can still be handled;
  • coverage on the larger test instances;
  • the total complexity of the selected features.

We also report the teacher's precision: the fraction of proposed features that end up selected, and the fraction of selected features that enumeration could not have reached at the baseline bound.

Four checks decide the outcome. First, on the published domains, the teacher-pool method must match 30/30 coverage while using a satisfiability problem at least ten times smaller. Second, on at least one domain or training size where equal-budget enumeration times out, our method must find a policy that generalizes. Third, the random-subset ablation must fall clearly below the teacher pool; otherwise the effect is only pool size. Fourth, the renamed-predicate results must hold within a few points of coverage.

If the first and third checks pass but the second fails, the honest conclusion is narrower: teacher proposals mainly reduce solver effort rather than extending what can be learned.