Abstract
Training robotic agents to follow natural language instructions is often hindered by the scarcity of high-quality, labeled trajectory data. This paper, presented at AAAI-2025, introduces Play Segmentation (PS), a probabilistic framework designed to extract and label task-specific segments from unannotated "play trajectories." By converting continuous, goal-agnostic behavior into clean training pairs, PS enables instruction-following policies to achieve performance levels comparable to those trained on twice the amount of manually labeled data.
Introduction
A major bottleneck in robotics is the data requirement for imitation learning. To teach a robot a task like "pick up the red block," researchers typically need thousands of paired examples of instructions and trajectories. While collecting such data is expensive, "play data"—long sequences of human-robot interaction without specific goals—is often abundant. However, play data is a continuous stream that lacks discrete boundaries and labels. Existing video segmentation tools often fail in this context because they require dense annotations or fixed-length actions. The authors address this by proposing a semi-supervised pipeline that uses a small amount of labeled data to unlock the potential of large, unlabeled play datasets.
Methodology: The Play Segmentation (PS) Model
The core innovation is a probabilistic model that factorizes the segmentation problem into two components:
- Labeling Model ($p_{\theta_{LM}}$): Predicts the appropriate natural language instruction for a given trajectory segment.
- Segmentation Model ($p_{\theta_{Seg}}$): Predicts whether a specific sequence of observations constitutes a valid, completed task.
To train the segmentation model without explicit negative examples, the authors employ a data augmentation strategy. They take ground-truth segments and intentionally shift or crop them to create "negative" samples, teaching the model to reject incomplete or noisy actions. At inference time, the model identifies the most likely sequence of segments across a long trajectory using a Dynamic Programming (DP) approach. To maintain computational efficiency over massive trajectories, a sliding window technique is applied.
Experimental Setup and Benchmarks
The researchers evaluated PS on two distinct environments:
- BabyAI: A grid-world environment requiring navigation to objects. It features high diversity despite relatively short trajectories.
- CALVIN: A complex simulator featuring a 7-DOF Franka Emika Panda robotic arm. Play trajectories in CALVIN can reach 30,000 steps and involve intricate manipulations such as opening drawers and moving blocks.
Key Results and Insights
The study yielded several critical findings regarding the role of data quality in imitation learning:
- Performance Gains: In the CALVIN environment, augmenting a 25% subset of labeled data with PS-extracted segments resulted in a policy that nearly matched the performance of one trained on 50% ground-truth data.
- Segmentation Accuracy: In BabyAI, PS achieved a task accuracy of 0.702, significantly outperforming computer vision baselines like UnLoc (0.583) and TriDet (0.261), which struggled to generalize to the messy boundaries of play data.
- Precision Over Recall: The PS model is intentionally conservative. While it may miss some valid segments (lower recall), the segments it identifies are high-quality (high precision). This is vital for imitation learning, where noisy or incorrectly labeled data can degrade policy performance.
- The Failure of Random Sampling: The paper demonstrates that adding randomly sampled segments from play data actually harms performance, emphasizing that how data is cropped is as important as the data itself.
Limitations and Future Work
Despite its success, the PS framework faces challenges with computational complexity due to the $O(T^3)$ nature of the DP approach. Additionally, the model can become biased if certain tasks are easier to identify than others, leading to an imbalanced augmented dataset. Future work may explore the use of Foundation Models (e.g., GPT-4V) to provide robust zero-shot labeling and segmentation, potentially removing the need for the initial labeled "start split."
Conclusion
Play Segmentation offers a powerful method for scaling robot training by making unannotated play data useful for instruction following. By prioritizing high-precision segmentation, the framework provides a reliable signal for downstream policies, effectively doubling the utility of limited human-labeled datasets.
Summary updated