Abstract
Due to burdensome data requirements, learning from demonstration often falls short of its promise to allow users to quickly and naturally program robots. Demonstrations are inherently ambiguous and incomplete, making correct generalization to unseen situations difficult without a large number of demonstrations in varying conditions. By contrast, humans are often able to learn complex tasks from a single demonstration (typically observations without action labels) by leveraging context learned over a lifetime. Inspired by this capability, our goal is to enable robots to perform one-shot learning of multi-step tasks from observation by leveraging auxiliary video data as context. Our primary contribution is a novel system that achieves this goal by: (1) using a single user-segmented demonstration to define the primitive actions that comprise a task, (2) localizing additional examples of these actions in unsegmented auxiliary videos via a metalearning-based approach, (3) using these additional examples to learn a reward function for each action, and (4) performing reinforcement learning on top of the inferred reward functions to learn action policies that can be combined to accomplish the task. We empirically demonstrate that a robot can learn multi-step tasks more effectively when provided auxiliary video, and that performance greatly improves when localizing individual actions, compared to learning from unsegmented videos.
Introduction
Learning from Demonstration (LfD) is a cornerstone of intuitive robot programming, yet it is frequently hampered by the "data bottleneck." While humans can observe a single video of a complex task and understand the underlying logic, robots typically require hundreds of demonstrations across diverse conditions to generalize effectively. This challenge is amplified in multi-step tasks, such as assembly or food preparation, where each sub-task involves distinct goals and object interactions.
Most existing Learning from Observation (LfO) methods—where robots learn from visual pixels without access to motor commands—are limited to simple, single-step actions. When applied to complex sequences, these methods often overfit to the single demonstration, failing to adapt to new environments. This paper proposes a framework that leverages unlabelled auxiliary videos (e.g., web-scale video data) to provide the necessary context for one-shot learning of multi-step robotic tasks.
Methodology
The proposed system utilizes a four-stage pipeline to bridge the context gap between a single demonstration and a robust control policy.
1. User-Segmented Demonstration
A human provides a single video demonstration of the multi-step task. The user manually segments this video, marking the boundaries between primitive actions (e.g., "pick up cup" vs. "pour water"). This segmentation defines the task's structure.
2. One-Shot Activity Localization
To gather more data without manual labeling, the system performs activity localization. Using meta-learning algorithms such as MAML and Reptile, the system uses the snippets from the user's demonstration to search through unsegmented auxiliary videos. It identifies and extracts additional examples of the same primitive actions, effectively building a task-specific dataset from unlabelled sources.
3. Reward Function Inference
For each sub-task, the system learns a reward function using a "shuffle-and-learn" approach. A neural network is trained to predict the temporal order of frames within the localized snippets. By learning to distinguish the correct sequence of an action, the network develops a surrogate reward function where progress toward the goal state is positively reinforced.
4. Reinforcement Learning (RL)
Finally, the robot employs Proximal Policy Optimization (PPO) to learn individual control policies for each sub-task based on the inferred rewards. These modular policies are then chained together to execute the full multi-step task.
Experimental Evaluation
The framework was evaluated using two primary datasets:
- Reacher Environment: A simulated two-joint robotic arm required to reach colored targets in specific sequences. This served as a testbed for the full pipeline from raw pixels to motor control.
- ActivityNet: A large-scale dataset of 100 human activities (e.g., playing golf). This was used to validate the one-shot localization algorithm's ability to handle the high variance and noise inherent in real-world video.
Key Results and Insights
The experiments revealed that modularization and auxiliary context are critical for one-shot success.
Localization and Generalization
The meta-learning approach for activity localization significantly outperformed standard classifiers. On the Reacher task, the meta-learning approach achieved a mean Intersection-over-Union (mIoU) of 0.84, whereas a standard classifier baseline only reached 0.48. This indicates that meta-learning is far more effective at identifying task primitives in unseen auxiliary data.
Policy Performance
The impact of auxiliary data on policy success was profound. In the multi-step reaching task, the robot achieved a success rate of approximately 85% when using localized auxiliary video. In stark contrast, when the robot attempted to learn from only the single user demonstration without auxiliary context, the success rate dropped to just 4%.
As shown in the performance curves, the "Ours (MAML Seg.)" approach approaches the performance of ground-truth (GT) rewards, while methods without proper segmentation or auxiliary data fail to converge effectively. This highlights that a single demonstration lacks the informational density required for generalization, a gap that can be filled by leveraging external video context.
Conclusion
This research demonstrates that robots can learn complex, multi-step tasks from a single observation by leveraging the vast amount of unlabelled video data available in the world. By localizing task primitives and inferring rewards through temporal ordering, the system bypasses the need for massive, hand-labeled datasets. Future work will focus on increasing robustness to extreme visual variance, such as shifts between third-person and egocentric perspectives, and incorporating multi-modal data like audio and text to further refine task understanding.
Summary updated