Learning Sensorimotor Primitives of Sequential Manipulation Tasks from Visual Demonstrations

Junchi Liang, Bowen Wen, Kostas E. Bekris, Abdeslam Boularias

Introduction

Teaching robots to perform complex, multi-step manipulation tasks—such as assembling furniture or performing maintenance—remains a significant challenge in robotics. Traditional imitation learning often struggles with "long-horizon" tasks for two primary reasons. First, models frequently focus on low-level motor skills (like grasping) without understanding how to chain them into a logical sequence. Second, most successful sequential models require heavy supervision, where humans must manually label every frame of a demonstration to indicate when one sub-task ends and the next begins. This "annotation bottleneck" makes scaling robot learning difficult and time-consuming.

Researchers at Rutgers University have addressed these challenges by developing a hierarchical neural network framework that learns to decompose complex tasks directly from raw video demonstrations. By utilizing an Expectation-Maximization (EM) approach, the system automatically segments videos into functional "primitives," allowing the robot to learn both high-level reasoning and low-level control without any manual data annotation.

Hierarchical Architecture

The proposed framework is organized into a three-layer hierarchy designed to bridge the gap between visual perception and motor execution:

  1. High-level Policy: This layer acts as an attention mechanism. It analyzes the scene and designates two objects as the primary actors: the "tool" (the object being manipulated) and the "target" (the destination or reference object). By focusing on this pair, the robot simplifies complex environments into manageable interactions.
  2. Intermediate-level Policy: Once the tool and target are identified, this layer predicts a specific 3D way-point (pose) that the robot must reach to progress through the sub-task.
  3. Low-level Policy: This layer translates the predicted way-point into precise motor commands for the robot's end-effector.

To ensure the robot understands the progression of a task, the system incorporates a Long Short-Term Memory (LSTM) unit. This memory component tracks completed steps, preventing the robot from repeating actions—such as repeatedly dipping a brush into paint—and ensuring it moves logically through the sequence.

Unsupervised Task Decomposition

The core innovation of this work is the use of an Expectation-Maximization (EM) algorithm combined with the Viterbi technique to solve the annotation bottleneck. During training, the robot is provided with videos of humans performing tasks but no labels regarding the sub-task structure.

The EM process works iteratively: the robot "guesses" the most likely sequence of tools, targets, and way-points that explain the human's movements in the video (the E-step), and then refines its policies to better match those inferred sequences (the M-step). Over time, the robot discovers the underlying logic of the task—such as "pick up the wrench before moving to the lug nut"—entirely through observation.

Experimental Results

The system was evaluated on four distinct tasks using both the PyBullet simulator and a real-world Kuka LBR robotic arm. The tasks included:

  • Painting: Dipping a brush in a bucket and painting a line.
  • Stacking: Arranging blocks into specific patterns.
  • Tire Removal: A complex sequence involving moving a wrench to four different lug nuts, rotating them, and returning to a central position.

Performance and Efficiency

In real-world testing, the unsupervised framework achieved a 100% success rate on the painting and stacking tasks. Notably, the performance of the unsupervised model was nearly identical to a "fully supervised" version of the same architecture, proving that the EM-based discovery of sub-tasks is highly effective.

Furthermore, the model significantly outperformed established imitation learning baselines, such as GAIL (Generative Adversarial Imitation Learning) and TACO. While these baselines often failed to complete the full sequence of sub-tasks, the proposed hierarchical approach maintained consistency across long horizons. Because the system operates on 6D object poses (tracked via se(3)-TrackNet) rather than raw pixels, it is also highly data-efficient, requiring only a few dozen demonstrations to master a task.

Conclusion and Future Work

This research demonstrates that robots can learn complex, multi-step manipulations from minimal human intervention. By automating task decomposition, the framework removes the need for tedious manual labeling.

However, some limitations remain. The current vision system relies on 3D CAD models for object tracking, and the "tool/target" abstraction may need to be expanded for tasks involving more than two interacting objects. Future work will focus on integrating model-free trackers to handle novel objects and developing more fluid, human-like trajectory planning between way-points.

Summary updated

Abstract

This work aims to learn how to perform complex robot manipulation tasks that are composed of several, consecutively executed low-level sub-tasks, given as input a few visual demonstrations of the tasks performed by a person. The sub-tasks consist of moving the robot's end-effector until it reaches a sub-goal region in the task space, performing an action, and triggering the next sub-task when a pre-condition is met. Most prior work in this domain has been concerned with learning only low-level tasks, such as hitting a ball or reaching an object and grasping it. This paper describes a new neural network-based framework for learning simultaneously low-level policies as well as high-level policies, such as deciding which object to pick next or where to place it relative to other objects in the scene. A key feature of the proposed approach is that the policies are learned directly from raw videos of task demonstrations, without any manual annotation or post-processing of the data. Empirical results on object manipulation tasks with a robotic arm show that the proposed network can efficiently learn from real visual demonstrations to perform the tasks, and outperforms popular imitation learning algorithms.
BibTeX
@inproceedings{liang2022learning,
  author = {Junchi Liang and Bowen Wen and Kostas E. Bekris and Abdeslam Boularias},
  title = {{Learning Sensorimotor Primitives of Sequential Manipulation Tasks from Visual Demonstrations}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2022},
  url = {https://openreview.net/forum?id=ItoT8pd_J},
}
Figure