Learning Sensorimotor Primitives of Sequential Manipulation Tasks from Visual Demonstrations

Junchi Liang, Bowen Wen, Kostas E. Bekris, Abdeslam Boularias

TL;DR

Researchers at Rutgers University have developed a hierarchical neural network framework that learns complex, multi-step robot manipulation tasks directly from raw video demonstrations. By using an Expectation-Maximization (EM) approach to automatically "segment" videos into sub-tasks, the system learns both high-level reasoning and low-level motor control without any manual data annotation.

What Problem Does This Paper Address?

Teaching robots to perform "long-horizon" tasks—like stacking blocks in a specific order or changing a tire—is notoriously difficult. Standard imitation learning usually falls into two traps:

  1. Low-level focus: It learns a single skill (like "grasping") but fails to chain them together.
  2. Heavy supervision: To learn complex sequences, humans usually have to manually label every frame of a video, telling the robot exactly when one sub-task ends and the next begins.

This paper addresses the "annotation bottleneck." The goal is to let a robot watch a few videos of a human performing a task and figure out the underlying logic (the "primitives") and the physical execution entirely on its own.

Methods

The authors propose a hierarchical neural network architecture organized into three layers:

  1. High-level Policy: Decides which two objects in the scene are currently important. It designates one as the "tool" (the object being moved) and the other as the "target" (where the tool is going).
  2. Intermediate-level Policy: Receives the tool/target pair and predicts a way-point (a specific 3D pose) that the robot needs to reach to complete the current sub-step.
  3. Low-level Policy: Translates that way-point into actual motor commands for the robot's end-effector.

The "Secret Sauce": Unsupervised Learning via EM Since the training videos don't come with labels for sub-tasks, the authors use an Expectation-Maximization (EM) algorithm combined with the Viterbi technique. The robot essentially "guesses" the most likely sequence of tools, targets, and way-points that explain the human's movement in the video, then refines its policies based on those guesses. This iterative process allows the robot to discover the structure of the task (e.g., "first pick up the brush, then dip it in the bucket") without being told.

Datasets

The researchers evaluated their system on four distinct tasks using both simulation (PyBullet) and a real-world Kuka LBR robot:

  • Painting: Dipping a brush in a bucket and painting a line on a surface.
  • Stacking (Pattern 1 & 2): Moving blocks into specific configurations.
  • Tire Removal: A complex sequence involving moving a wrench to four different lug nuts, rotating them, and then pulling the wrench to the center.

Data was collected using an Intel RealSense 415 camera to record human demonstrations, with 6D object poses tracked in real-time using se(3)-TrackNet.

Key Insights and Ideas

  • The Power of Attention: By forcing the high-level policy to pick a "tool" and a "target," the researchers effectively implemented an attention mechanism. Instead of the robot trying to process the entire scene at once, it focuses only on the relationship between two relevant objects, drastically simplifying the learning problem.
  • Memory Matters: The system uses an LSTM (Long Short-Term Memory) unit to keep track of what has already been accomplished. This prevents the robot from getting stuck in loops (e.g., dipping the brush in paint over and over).
  • Data Efficiency: Unlike end-to-end "pixels-to-actions" models that require thousands of trials, this method learns effectively from just a few dozen demonstrations because it operates on 6D object poses rather than raw image pixels.

Key Results

  • Real-World Success: On the physical Kuka robot, the proposed unsupervised method achieved a 100% success rate (5/5) on painting and stacking tasks.
  • Outperforming Baselines: It significantly outperformed popular algorithms like GAIL (Generative Adversarial Imitation Learning) and TACO, which often failed to complete the full sequence of sub-tasks.
  • Minimal Gap: The "unsupervised" version of their model performed nearly as well as a "supervised" version where humans provided ground-truth labels, proving that their EM-based discovery of sub-tasks is highly effective.

Limitations and Future Work

  • CAD Dependency: The current vision system requires 3D CAD models of the objects to track their 6D poses. The authors note that future work could involve using model-free trackers to handle novel, unknown objects.
  • Task Complexity: While the tool/target abstraction is powerful, it might struggle with tasks involving more than two objects interacting simultaneously (e.g., using a screwdriver to hold a screw while a second tool tightens a nut).
  • Trajectory Smoothing: While the robot reaches the way-points, the motions between them are currently simple; integrating more fluid, "human-like" motion planning could be a next step.

Summary updated

Abstract

This work aims to learn how to perform complex robot manipulation tasks that are composed of several, consecutively executed low-level sub-tasks, given as input a few visual demonstrations of the tasks performed by a person. The sub-tasks consist of moving the robot's end-effector until it reaches a sub-goal region in the task space, performing an action, and triggering the next sub-task when a pre-condition is met. Most prior work in this domain has been concerned with learning only low-level tasks, such as hitting a ball or reaching an object and grasping it. This paper describes a new neural network-based framework for learning simultaneously low-level policies as well as high-level policies, such as deciding which object to pick next or where to place it relative to other objects in the scene. A key feature of the proposed approach is that the policies are learned directly from raw videos of task demonstrations, without any manual annotation or post-processing of the data. Empirical results on object manipulation tasks with a robotic arm show that the proposed network can efficiently learn from real visual demonstrations to perform the tasks, and outperforms popular imitation learning algorithms.
BibTeX
@inproceedings{liang2022learning,
  author = {Junchi Liang and Bowen Wen and Kostas E. Bekris and Abdeslam Boularias},
  title = {{Learning Sensorimotor Primitives of Sequential Manipulation Tasks from Visual Demonstrations}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2022},
  url = {https://openreview.net/forum?id=ItoT8pd_J},
}

Loading related content...

Figure