DeepSym: Deep Symbol Generation and Rule Learning for Planning from Unsupervised Robot Interaction

Alper Ahmetoglu, M. Yunus Seker, Justus H. Piater, Erhan Öztop, Emre UgurJAIR 2022 · DOI · OpenReview

TL;DR

This paper introduces DeepSym, a novel method for robots to learn symbolic representations and rules directly from raw sensorimotor interaction data. This allows robots to autonomously build the knowledge needed for complex planning in open-ended environments, overcoming the limitations of manually defined symbols.

What Problem Does This Paper Address?

Robots excel at low-level control but struggle with high-level reasoning and planning, especially in dynamic or unknown environments. This is largely due to the "symbol grounding problem": bridging the gap between continuous sensory input and discrete, meaningful symbols that AI planners can use. Traditionally, this requires humans to manually define these symbols and rules, which is not scalable for robots operating in diverse, real-world scenarios. DeepSym aims to automate this process by learning these symbols and rules from the robot's own interactions with its environment.

Methods

DeepSym employs a deep encoder-decoder neural network architecture.

  1. Interaction & Data Collection: A robot interacts with objects using a predefined set of actions (e.g., poking, stacking) and records the observed state (depth image) and resulting effect (change in object position/force).
  2. Symbol Formation (Encoder-Decoder): A predictive encoder-decoder network is trained. The encoder takes an object's state (image) and an action as input and outputs a discrete latent representation (symbol). The decoder takes this symbol and action to predict the resulting effect. A key innovation is the use of a binary bottleneck layer in the encoder, which forces the network to learn discrete, symbolic representations of objects and their properties.
  3. Rule Learning (Decision Tree): The learned symbolic representations and predicted effects are used to train a decision tree. This tree captures the probabilistic relationships between object symbols, actions, and effects, effectively learning rules.
  4. Planning (PDDL): The learned rules are translated into Probabilistic Planning Domain Definition Language (PPDDL) operators. This allows standard AI planners to generate plans for achieving goals.

Datasets

The paper uses two main environments:

  • Simulated Robotic Manipulator Environment: A custom setup with a UR10 robot arm, various objects (cups, cylinders, spheres, cubes), and a Kinect sensor. The robot performs actions like poking and stacking.
  • MNIST 8-puzzle: A benchmark adapted from existing work. Here, the "state" is a raw image of the puzzle board, and actions are "slide" operations. This demonstrates the method's applicability beyond robotics.

Key Insights and Ideas

  • Action-Grounded Symbols: Symbols learned by DeepSym are not just static object properties but are intrinsically linked to actions and their effects. This makes them more relevant for planning.
  • Binary Bottleneck is Key: Using a binary bottleneck layer in the encoder is crucial for forcing the network to learn discrete, interpretable symbols, which are more amenable to rule extraction than continuous latent spaces.
  • Unsupervised Discovery: The system autonomously discovers object properties like 'rollable', 'insertable', and relational concepts like 'larger-than' without human supervision.
  • Progressive Learning: The architecture is designed to allow for progressive learning, where previously learned abstractions can be used as input for learning more complex ones.
  • Domain Generality: The method shows promise by successfully applying to both a robotic manipulation task and a classic AI puzzle domain.

Key Results

  • Robotics: DeepSym successfully learned symbolic representations that enabled a robot to generate effective plans for tasks like building towers of a desired height. It achieved 95% planning accuracy compared to an alternative method (OCEC) with 25% accuracy.
  • 8-Puzzle: The system learned symbols that allowed it to generate plans to move the empty tile to arbitrary positions, demonstrating its ability to handle complex state spaces.
  • Object Categorization: In the robotic domain, DeepSym achieved 98.5% accuracy in categorizing objects, outperforming an autoencoder-based baseline (60.6% accuracy).

Limitations and Future Work

  • Local vs. Global State: In the 8-puzzle, the current formulation primarily captures local state information (e.g., the empty tile and its neighbors) rather than the global state of the entire puzzle. This can lead to plans that correctly position the empty tile but disregard other tiles.
  • Scalability of Bottleneck: For very large state spaces (like the 15-puzzle), a larger bottleneck size might be necessary to encode all relevant states, which can increase computational cost.
  • Future Work: The authors suggest augmenting perceptual capabilities, using graph neural networks for more complex object relations, and applying the principles to less-constrained, general-purpose robot scenarios. They also plan to extend the planner to utilize numeric values associated with effect clusters instead of just atomic effects.

Summary updated

Abstract

Symbolic planning and reasoning are powerful tools for robots tackling complex tasks. However, the need to manually design the symbols restrict their applicability, especially for robots that are expected to act in open-ended environments. Therefore symbol formation and rule extraction should be considered part of robot learning, which, when done properly, will offer scalability, flexibility, and robustness. Towards this goal, we propose a novel general method that finds action-grounded, discrete object and effect categories and builds probabilistic rules over them for non-trivial action planning. Our robot interacts with objects using an initial action repertoire that is assumed to be acquired earlier and observes the effects it can create in the environment. To form action-grounded object, effect, and relational categories, we employ a binary bottleneck layer in a predictive, deep encoderdecoder network that takes the image of the scene and the action applied as input, and generates the resulting effects in the scene in pixel coordinates. After learning, the binary latent vector represents action-driven object categories based on the interaction experience of the robot. To distill the knowledge represented by the neural network into rules useful for symbolic reasoning, a decision tree is trained to reproduce its decoder function. Probabilistic rules are extracted from the decision paths of the tree and are represented in the Probabilistic Planning Domain Definition Language (PPDDL), allowing off-the-shelf planners to operate on the knowledge extracted from the sensorimotor experience of the robot. The deployment of the proposed approach for a simulated robotic manipulator enabled the discovery of discrete representations of object properties such as ‘rollable’ and ‘insertable’. In turn, the use of these representations as symbols allowed the generation of effective plans for achieving goals, such as building towers of the desired height, demonstrating the effectiveness of the approach for multi-step object manipulation. Finally, we demonstrate that the system is not only restricted to the robotics domain by assessing its applicability to the MNIST 8-puzzle domain in which learned symbols allow for the generation of plans that move the empty tile into any given position.
BibTeX
@article{ahmetoglu2022deepsym,
  author = {Alper Ahmetoglu and M. Yunus Seker and Justus H. Piater and Erhan Öztop and Emre Ugur},
  title = {{DeepSym: Deep Symbol Generation and Rule Learning for Planning from Unsupervised Robot Interaction}},
  journal = {J. Artif. Intell. Res.},
  year = {2022},
  doi = {10.1613/jair.1.13754},
}
Figure