Introduction
Teaching robots to perform complex, multi-step manipulation tasks—such as assembling furniture or performing maintenance—remains a significant challenge in robotics. Traditional imitation learning often struggles with "long-horizon" tasks for two primary reasons. First, models frequently focus on low-level motor skills (like grasping) without understanding how to chain them into a logical sequence. Second, most successful sequential models require heavy supervision, where humans must manually label every frame of a demonstration to indicate when one sub-task ends and the next begins. This "annotation bottleneck" makes scaling robot learning difficult and time-consuming.
Researchers at Rutgers University have addressed these challenges by developing a hierarchical neural network framework that learns to decompose complex tasks directly from raw video demonstrations. By utilizing an Expectation-Maximization (EM) approach, the system automatically segments videos into functional "primitives," allowing the robot to learn both high-level reasoning and low-level control without any manual data annotation.
Hierarchical Architecture
The proposed framework is organized into a three-layer hierarchy designed to bridge the gap between visual perception and motor execution:
- High-level Policy: This layer acts as an attention mechanism. It analyzes the scene and designates two objects as the primary actors: the "tool" (the object being manipulated) and the "target" (the destination or reference object). By focusing on this pair, the robot simplifies complex environments into manageable interactions.
- Intermediate-level Policy: Once the tool and target are identified, this layer predicts a specific 3D way-point (pose) that the robot must reach to progress through the sub-task.
- Low-level Policy: This layer translates the predicted way-point into precise motor commands for the robot's end-effector.
To ensure the robot understands the progression of a task, the system incorporates a Long Short-Term Memory (LSTM) unit. This memory component tracks completed steps, preventing the robot from repeating actions—such as repeatedly dipping a brush into paint—and ensuring it moves logically through the sequence.
Unsupervised Task Decomposition
The core innovation of this work is the use of an Expectation-Maximization (EM) algorithm combined with the Viterbi technique to solve the annotation bottleneck. During training, the robot is provided with videos of humans performing tasks but no labels regarding the sub-task structure.
The EM process works iteratively: the robot "guesses" the most likely sequence of tools, targets, and way-points that explain the human's movements in the video (the E-step), and then refines its policies to better match those inferred sequences (the M-step). Over time, the robot discovers the underlying logic of the task—such as "pick up the wrench before moving to the lug nut"—entirely through observation.
Experimental Results
The system was evaluated on four distinct tasks using both the PyBullet simulator and a real-world Kuka LBR robotic arm. The tasks included:
- Painting: Dipping a brush in a bucket and painting a line.
- Stacking: Arranging blocks into specific patterns.
- Tire Removal: A complex sequence involving moving a wrench to four different lug nuts, rotating them, and returning to a central position.
Performance and Efficiency
In real-world testing, the unsupervised framework achieved a 100% success rate on the painting and stacking tasks. Notably, the performance of the unsupervised model was nearly identical to a "fully supervised" version of the same architecture, proving that the EM-based discovery of sub-tasks is highly effective.
Furthermore, the model significantly outperformed established imitation learning baselines, such as GAIL (Generative Adversarial Imitation Learning) and TACO. While these baselines often failed to complete the full sequence of sub-tasks, the proposed hierarchical approach maintained consistency across long horizons. Because the system operates on 6D object poses (tracked via se(3)-TrackNet) rather than raw pixels, it is also highly data-efficient, requiring only a few dozen demonstrations to master a task.
Conclusion and Future Work
This research demonstrates that robots can learn complex, multi-step manipulations from minimal human intervention. By automating task decomposition, the framework removes the need for tedious manual labeling.
However, some limitations remain. The current vision system relies on 3D CAD models for object tracking, and the "tool/target" abstraction may need to be expanded for tasks involving more than two interacting objects. Future work will focus on integrating model-free trackers to handle novel objects and developing more fluid, human-like trajectory planning between way-points.
Summary updated