Target Languages (vs. Inductive Biases) for Learning to Act and Plan

Hector GeffnerAAAI 2022 · DOI

Introduction

In the current era of Artificial Intelligence, Deep Learning (DL) and Reinforcement Learning (RL) have achieved remarkable success in domains like Chess and Go. However, these systems often exhibit a lack of robustness, crumbling when faced with situations slightly outside their training distribution. In his paper, "Target Languages (vs. Inductive Biases) for Learning to Act and Plan," Hector Geffner (AAAI-22) argues that the path to human-like generalization lies not in more data or architectural tweaks, but in learning representations expressed in formal target languages with well-defined semantics.

The Problem: Out-of-Distribution Generalization

The central challenge addressed is out-of-distribution (OOD) generalization. A neural network trained to solve a $5 \times 5$ gridworld puzzle often fails on a $10 \times 10$ version, even if the underlying rules remain identical. The standard response in the DL community is to introduce "inductive biases"—assumptions built into the neural architecture, such as the spatial invariance of a Convolutional Neural Network (CNN). Geffner argues that these biases are often "vague" and lack explicit evaluation. They provide a "how" (a learning mechanism) but fail to define a clear "what" (the resulting knowledge).

Conversely, symbolic AI has long utilized powerful languages for generalization, such as first-order logic, but these were traditionally hand-crafted. Geffner identifies the need to learn these symbolic representations directly from data, combining the scaling power of neural networks with the structural robustness of logic.

Methodology: A Language-Based Approach

Geffner proposes a shift toward a "language-based approach to representation learning." Rather than allowing features to emerge as opaque weights, we define a target language—the "needle" we are searching for—and formulate learning as a search for the simplest description in that language that fits the observed data. This approach highlights three specific applications:

  1. General Action Models: Learning the "physics" of an environment through "action schemas" (e.g., a pick-up action requires an empty hand and results in holding an object).
  2. General Policies: Learning rules that apply across any instance of a problem. For example, a policy for a delivery task might be: "If not holding a package, move toward the nearest one."
  3. Policy Sketches: Learning to decompose complex goals into subgoals using a formal language that defines how state features should change.

Technically, this transforms learning into a combinatorial optimization problem (similar to Weighted Max-SAT), where the objective is to find the most compact description, as simplicity in a formal language serves as a strong proxy for generalization.

Benchmarks and Applications

The paper utilizes several classic and modern AI benchmarks to illustrate these concepts, most notably Minigrid, a popular gridworld environment used to test an agent's ability to follow complex, multi-step instructions.

A complex Minigrid environment featuring multiple rooms, keys, and doors. Such environments serve as benchmarks for testing an agent's ability to generalize symbolic rules across varying grid configurations.

Other benchmarks include Delivery (a simplified gridworld for general policies) and Classical Planning Domains like BlocksWorld and the Tower of Hanoi. These domains demonstrate that a model learned from a small instance (e.g., 3 disks in Hanoi) can be proven correct for instances of any size.

Key Insights

  • Compactness = Generalization: In a formal language, a "short" program is more likely to represent the true underlying logic of a domain, providing a stronger learning signal than simple loss minimization.
  • The "Needle" Metaphor: Quoting Judea Pearl, Geffner notes that "it helps to know what a needle looks like." Defining a target language provides a template for successful abstraction.
  • Semantics and Transparency: Unlike neural features, representations in a target language have explicit semantics, making the resulting AI more transparent and easier to debug.
  • Hybrid Synergy: Geffner suggests that symbolic AI provides the account of what is to be learned, while neural networks provide the account of how those representations can be extracted from raw data.

Results and Future Outlook

While primarily a position paper, it references significant results: systems can now learn first-order action schemas (like PDDL) from state transition graphs, and general policies for the "Delivery" domain can solve instances in linear time—a feat standard RL struggles to achieve.

However, challenges remain. The Symbol Grounding Problem—moving from raw pixels to the predicates used in these languages—is still a research frontier. Furthermore, searching the space of all possible symbolic descriptions is computationally expensive. Future work must focus on more efficient optimization techniques and the development of hybrid architectures where neural perception layers feed directly into symbolic reasoning modules.

Summary updated

BibTeX
@inproceedings{geffner2022target,
  author = {Hector Geffner},
  title = {{Target Languages (vs. Inductive Biases) for Learning to Act and Plan}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2022},
  doi = {10.1609/aaai.v36i11.21497},
}
Figure