In traditional reinforcement learning, an agent’s understanding of the world is often tied to a fixed coordinate system or a specific number of objects. If you train a robot to stack three blocks, it may become completely lost when presented with four. This "scale gap" occurs because standard models learn to map specific state features to actions, rather than learning the underlying logic of the task.
Relational generalization addresses this by shifting the focus from specific object identities to the relationships between them. By representing policies through relational logic or graph-based structures, researchers have developed agents that can learn in small, simple environments and apply that knowledge to massive, complex ones without any additional training.
The Power of Relational Bias
The core intuition behind this research is that the "rules" of a domain—like how to clear a path or stack a tower—do not change just because the environment gets bigger. Alan Fern et al. in Approximate Policy Iteration with a Policy Language Bias propose using a "relational language bias." Instead of learning that "Block A is at position X," the agent learns a rule like "if a block is on the target, move it to the table."
Because these rules are defined by properties and relationships rather than specific identifiers, they are inherently scale-invariant. As Kurt Driessens et al. demonstrate in Speeding Up Relational Reinforcement Learning through the Use of an Incremental First Order Decision Tree Learner, a policy expressed as a first-order decision tree can be applied to a problem with any number of objects. The logic remains valid whether there are five blocks or five hundred.
Learning from Minimal Examples
One of the most striking results of this approach is the ability to achieve "complete generalization" from tiny amounts of data. Charles Gretton et al. show in Exploiting First-Order Regression in Inductive Policy Selection that by extracting the logical essence of a task from minimal examples, an inductive learner can produce policies that are mathematically optimal for problems of arbitrary scale.
This efficiency extends to complex planning tasks. Tomás de la Rosa et al. developed the ROLLER system, which uses relational decision trees to capture "helpful contexts" for planners. By training on small instances, the system learns search control knowledge that allows it to solve massive problems—such as 100-block tasks—orders of magnitude faster than traditional planners (Scaling up Heuristic Planning with Relational Decision Trees).
Bridging Logic and Deep Learning
Modern research has integrated these relational concepts into neural architectures. The challenge is creating a neural network that can accept an arbitrary number of inputs.
- Action Schema Networks (ASNets): Sam Toyer et al. introduced a neural architecture that mirrors the relational structure of a domain. By tying weights to action types rather than specific objects, ASNets can generalize from small training instances to significantly larger problem sizes (Action Schema Networks: Generalised Policies With Deep Learning).
- Symbolic Networks (SYMNET): The authors of Symbolic Network: Generalized Neural Policies for Relational MDPs take this further by converting symbolic logic into instance graphs. This allows a single forward pass of a neural network to solve brand-new, larger versions of a problem without retraining.
- Neural Logic RL: Zhengyao Jiang et al. propose NLRL, which represents policies using a differentiable logic engine. This results in agents that are not only scale-invariant but also human-readable (Neural Logic Reinforcement Learning).
Generalizing Numeric and Predictive States
Relational generalization is not limited to discrete objects; it also applies to numeric reasoning and observable sequences. Ryan Xiao Wang et al. demonstrate that prioritizing the logical interactions between numeric variables (like comparisons) over raw values allows policies to generalize even when quantitative constraints are proportionally increased (Learning Generalised Policies for Numeric Planning).
Furthermore, David Wingate et al. show that relational policies can be grounded in Predictive State Representations (PSRs). By defining policies through observable tests rather than hidden states, agents can manipulate stacks of objects regardless of whether the environment contains a few items or hundreds (Relational Knowledge with Predictive State Representations).
Why It Works: The Lifted Perspective
The unifying mechanism across these papers is lifting. Instead of grounding a problem into a massive, specific state space, these methods reason at a "lifted" level—identifying structural patterns that exist across all instances of a domain. As Daniel Gnad 0001 et al. argue, training on abstract relational features allows models to predict action utility in environments so large they would otherwise crash a standard planner (Learning How to Ground a Plan - Partial Grounding in Classical Planning).
By focusing on the topology of the problem rather than its geometry, relational generalization provides a path toward agents that can truly understand the structure of their tasks, enabling them to scale from the simple to the complex with mathematical grace.
Key Foundational Work
Approximate Policy Iteration with a Policy Language Bias (2003) — contributed 2 concept elements to this direction.
Speeding Up Relational Reinforcement Learning through the Use of an Incremental First Order Decision Tree Learner (2001) — contributed 1 concept elements to this direction.