TL;DR
This paper introduces DeepSym, a novel method for robots to learn symbolic representations and rules directly from raw sensorimotor interaction data. This allows robots to autonomously build the knowledge needed for complex planning in open-ended environments, overcoming the limitations of manually defined symbols.
What Problem Does This Paper Address?
Robots excel at low-level control but struggle with high-level reasoning and planning, especially in dynamic or unknown environments. This is largely due to the "symbol grounding problem": bridging the gap between continuous sensory input and discrete, meaningful symbols that AI planners can use. Traditionally, this requires humans to manually define these symbols and rules, which is not scalable for robots operating in diverse, real-world scenarios. DeepSym aims to automate this process by learning these symbols and rules from the robot's own interactions with its environment.
Methods
DeepSym employs a deep encoder-decoder neural network architecture.
- Interaction & Data Collection: A robot interacts with objects using a predefined set of actions (e.g., poking, stacking) and records the observed state (depth image) and resulting effect (change in object position/force).
- Symbol Formation (Encoder-Decoder): A predictive encoder-decoder network is trained. The encoder takes an object's state (image) and an action as input and outputs a discrete latent representation (symbol). The decoder takes this symbol and action to predict the resulting effect. A key innovation is the use of a binary bottleneck layer in the encoder, which forces the network to learn discrete, symbolic representations of objects and their properties.
- Rule Learning (Decision Tree): The learned symbolic representations and predicted effects are used to train a decision tree. This tree captures the probabilistic relationships between object symbols, actions, and effects, effectively learning rules.
- Planning (PDDL): The learned rules are translated into Probabilistic Planning Domain Definition Language (PPDDL) operators. This allows standard AI planners to generate plans for achieving goals.
Datasets
The paper uses two main environments:
- Simulated Robotic Manipulator Environment: A custom setup with a UR10 robot arm, various objects (cups, cylinders, spheres, cubes), and a Kinect sensor. The robot performs actions like poking and stacking.
- MNIST 8-puzzle: A benchmark adapted from existing work. Here, the "state" is a raw image of the puzzle board, and actions are "slide" operations. This demonstrates the method's applicability beyond robotics.
Key Insights and Ideas
- Action-Grounded Symbols: Symbols learned by DeepSym are not just static object properties but are intrinsically linked to actions and their effects. This makes them more relevant for planning.
- Binary Bottleneck is Key: Using a binary bottleneck layer in the encoder is crucial for forcing the network to learn discrete, interpretable symbols, which are more amenable to rule extraction than continuous latent spaces.
- Unsupervised Discovery: The system autonomously discovers object properties like 'rollable', 'insertable', and relational concepts like 'larger-than' without human supervision.
- Progressive Learning: The architecture is designed to allow for progressive learning, where previously learned abstractions can be used as input for learning more complex ones.
- Domain Generality: The method shows promise by successfully applying to both a robotic manipulation task and a classic AI puzzle domain.
Key Results
- Robotics: DeepSym successfully learned symbolic representations that enabled a robot to generate effective plans for tasks like building towers of a desired height. It achieved 95% planning accuracy compared to an alternative method (OCEC) with 25% accuracy.
- 8-Puzzle: The system learned symbols that allowed it to generate plans to move the empty tile to arbitrary positions, demonstrating its ability to handle complex state spaces.
- Object Categorization: In the robotic domain, DeepSym achieved 98.5% accuracy in categorizing objects, outperforming an autoencoder-based baseline (60.6% accuracy).
Limitations and Future Work
- Local vs. Global State: In the 8-puzzle, the current formulation primarily captures local state information (e.g., the empty tile and its neighbors) rather than the global state of the entire puzzle. This can lead to plans that correctly position the empty tile but disregard other tiles.
- Scalability of Bottleneck: For very large state spaces (like the 15-puzzle), a larger bottleneck size might be necessary to encode all relevant states, which can increase computational cost.
- Future Work: The authors suggest augmenting perceptual capabilities, using graph neural networks for more complex object relations, and applying the principles to less-constrained, general-purpose robot scenarios. They also plan to extend the planner to utilize numeric values associated with effect clusters instead of just atomic effects.
Summary updated