Learning visual policies for building 3D shape categories

Alexander Pashevich, Igor Kalevatykh, Ivan Laptev, Cordelia SchmidIROS 2020 · OpenReview

TL;DR

This paper introduces a novel method for robots to learn how to build 3D shapes from various building blocks, even those they haven't seen before. By first learning assembly plans in a simulated "state space" and then translating these into visual policies for a real robot, they achieve impressive success rates in constructing complex shapes like arches.

What Problem Does This Paper Address?

Robots are still far from human-level dexterity and planning capabilities, especially in complex manipulation and assembly tasks. While previous work often focused on building specific object instances from a fixed set of parts, this paper tackles a more challenging problem: building different objects within a shape category using varying sets of primitives. This requires generalization beyond specific examples, a crucial step towards more adaptable robotic systems. The challenge lies in learning these complex assembly sequences from visual input alone, without relying on explicit 3D scene reconstruction or extensive human demonstrations.

Methods

The authors propose a two-stage approach:

  1. State Space Learning & Discovery:
    • Disassembly Procedure: They introduce a novel "unmaking" or disassembly process. By taking an existing object and reversing its assembly steps, they generate valid action sequences. This is key to discovering new object instances and their assembly plans without needing pre-defined demonstrations.
  2. State Policy: Using these disassembly trajectories, they learn a state-value function that guides a policy to discover new object instances within a shape category. This process is iterated to expand the set of learned assembly plans.
  3. Visual Policy Learning (Observation Space):
    • Simulation-to-Real Transfer: The assembly trajectories learned in the state space are used to generate realistic visual observations (depth images) from a simulator.
  4. Heatmap Representation: To handle the multi-modal nature of actions (e.g., multiple valid places to put a block), they use a heatmap representation for the output of their visual policies. This allows the policy to predict multiple potential actions simultaneously.
  5. Behavioral Cloning (BC): A Convolutional Neural Network (CNN), specifically a HourGlass architecture, is trained using Behavioral Cloning to map visual observations to these action heatmaps.
  6. Sim2Real Augmentation: To bridge the gap between simulation and the real world, they employ sim2real augmentation techniques.

Datasets

  • Simulated Data: The core of the training relies on simulated data generated from the authors' proposed disassembly and state-space learning procedures. This includes state-action pairs and rendered visual observations (depth images) with corresponding action heatmaps.
  • Real Robot Data: Evaluation is performed on a real UR5 robot. The paper mentions using a Microsoft Kinect-2 camera for depth sensing and color segmentation masks.

Key Insights and Ideas

  • Disassembly as a Generator: The "unmaking" or disassembly procedure is a clever way to generate training data (assembly trajectories) for complex tasks without requiring explicit human demonstrations. This is a significant contribution to data-efficient learning for robotics.
  • Heatmaps for Multi-Modal Actions: Using heatmaps to represent action outputs is crucial for handling the inherent ambiguity in robotic manipulation tasks, where multiple actions might be valid in a given state. This allows for more robust policy learning.
  • Generalization to Shape Categories: The method's ability to learn policies for entire shape categories, rather than just specific instances, is a major step towards more flexible robotic assembly.
  • Sim2Real Effectiveness: The successful transfer of policies trained in simulation to a real robot, even with unseen primitives, highlights the effectiveness of their sim2real augmentation and overall approach.

Key Results

  • State Space Efficiency: In simulation, their method significantly outperforms random exploration and MCTS baselines in terms of steps required to build arches, achieving near-oracle performance (e.g., 6.9 steps for a 5U arch vs. 1.6e4 for MCTS).
  • Visual Policy Performance:
    • Achieved high success rates (up to 95%) on a real UR5 robot for building arches.
  • Demonstrated robustness to unseen primitives, generalizing to shapes not encountered during training.
  • Showed reactive capabilities, successfully re-assembling objects even when additional primitives were introduced to the scene.
  • The HourGlass architecture with heatmaps outperformed a ResNet-18 baseline on more complex tower-building tasks where identical primitives were present.

Limitations and Future Work

  • Complexity of Disassembly: The authors acknowledge that the complexity of the unmake procedure grows exponentially with the number of primitives. They suggest future work could involve sampling disassembly trajectories to address more complex tasks.
  • Primitive Limitations: While the method generalizes to new shapes, its performance is still impacted when primitives are significantly different from those used during training (e.g., stones vs. blocks). Further improvements could involve training with more diverse shapes.
  • Error Correction: The paper mentions a post-processing step to correct predicted locations using depth images, indicating that raw policy outputs can still have errors.
  • Physically Irreversible Actions: The current disassembly approach might not be directly applicable to all tasks with physically irreversible actions. Future work could explore learning appropriate backward models in state space.

Summary updated

Abstract

Manipulation and assembly tasks require non-trivial planning of actions depending on the environment and the final goal. Previous work in this domain often assembles particular instances of objects from known sets of primitives. In contrast, we aim to handle varying sets of primitives and to construct different objects of a shape category. Given a single object instance of a category, e.g. an arch, and a binary shape classifier, we learn a visual policy to assemble other instances of the same category. In particular, we propose a disassembly procedure and learn a state policy that discovers new object instances and their assembly plans in state space. We then render simulated states in the observation space and learn a heatmap representation to predict alternative actions from a given input image. To validate our approach, we first demonstrate its efficiency for building object categories in state space. We then show the success of our visual policies for building arches from different primitives. Moreover, we demonstrate (i) the reactive ability of our method to re-assemble objects using additional primitives and (ii) the robust performance of our policy for unseen primitives resembling building blocks used during training. Our visual assembly policies are trained with no real images and reach up to 95% success rate when evaluated on a real robot.
BibTeX
@inproceedings{pashevich2020learning,
  author = {Alexander Pashevich and Igor Kalevatykh and Ivan Laptev and Cordelia Schmid},
  title = {{Learning visual policies for building 3D shape categories}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2020},
  url = {https://openreview.net/forum?id=8bLPuv1AXJTr},
}
Figure