TL;DR
Demo-SCORE (Curating Demonstrations using Online Experience) is a self-curation framework that allows robots to automatically filter out "bad" or unreliable demonstrations from large, messy datasets. By training a classifier on its own trial-and-error rollouts, the robot learns to identify which human strategies are actually reproducible, leading to a 15–35% boost in success rates without any manual data labeling.
What Problem Does This Paper Address?
Imitation learning (IL) relies on the quality of human demonstrations. However, large-scale datasets are often "heterogeneous"—meaning they contain a mix of different strategies, some of which might be suboptimal, rare, or simply too difficult for a robot's specific hardware to replicate reliably.
For example, a human might pick up a spoon by the very tip of the handle. While a human can do this easily, a robot's gripper might find this strategy unstable, leading to frequent drops. If a robot is trained on all these demonstrations indiscriminately, it may attempt these "unreliable" strategies at test time and fail. Manually sifting through thousands of videos to find these edge cases is prohibitively expensive and slow. The authors ask: Can a robot use its own experience to figure out which demonstrations it should actually follow?
Methods
The authors propose Demo-SCORE, a four-step pipeline for automated data curation:
- Train an Initial Policy: Train a base policy (using architectures like Diffusion Policy or ACT) on the entire, unfiltered dataset. The goal here is to capture the full distribution of behaviors, even the bad ones.
- Collect Online Rollouts: Run the initial policy in the environment at various stages of training (checkpoints). Each attempt (rollout) is automatically labeled as a "success" or "failure" based on a binary indicator (e.g., did the peg go in the hole?).
- Train a Data Quality Classifier: Train a classifier (a simple MLP) to distinguish between the successful and failed rollouts. Crucially, they use cross-validation across different training checkpoints to ensure the classifier doesn't just overfit to one specific version of the robot's behavior.
- Filter and Retrain: Use the trained classifier to "score" the original human demonstrations. Demonstrations that look like the robot's failed attempts are discarded. Finally, a new, "final" policy is trained on this curated, high-quality subset.
Datasets
The researchers evaluated Demo-SCORE across both simulated and real-world environments:
- Simulation (Robosuite & ALOHA): Bimanual peg insertion and a square peg task. They intentionally created "lopsided" mixtures of scripted and human data to test the filter's robustness.
- Real-World (ALOHA): Five diverse manipulation tasks:
- Placing a spoon in a rack.
- Transferring a chocolate bar between plates.
- Picking up a Sharpie from a cluttered scene.
- Picking up a strawberry.
- The Jellybean Task: A complex, long-horizon task involving five sub-steps (retrieving a cup, filling it with jellybeans, etc.).
Key Insights and Ideas
- The Robot is the Best Judge of Itself: Humans often can't tell which strategies will be hard for a robot. Demo-SCORE grounds data curation in the robot's actual embodiment and limitations.
- Failure is Information: Most IL methods only look at successful demonstrations. Demo-SCORE uses the robot's failures during training to define what "bad" data looks like.
- Cross-Validation is Critical: A classifier trained on rollouts from just one policy checkpoint tends to overfit. By using rollouts from multiple points in the training process, the classifier learns a more general representation of "reliability" that applies better to the original human data.
- Granularity Matters: The method works whether you filter entire trajectories or just specific "chunks" of a long task, making it flexible for complex, multi-stage behaviors.
Key Results
- Performance Boost: Policies trained with Demo-SCORE achieved 15–35% higher absolute success rates compared to policies trained on the full, unfiltered datasets.
- Real-World Success: In real-world ALOHA tasks, Demo-SCORE improved success rates by nearly 30% on average. On the difficult Jellybean task, the base policy had a 0% full-task success rate, while the Demo-SCORE curated policy reached 20%.
- Efficiency: The method is highly sample-efficient. Experiments showed that as few as 10 rollouts per checkpoint (50 total) were enough to train an effective classifier.
- Comparison: Demo-SCORE consistently outperformed prior "Auto-IL" methods and loss-based weighting schemes, particularly in scenarios with highly varied human strategies.
Limitations and Future Work
- Initial Policy Quality: Demo-SCORE assumes the initial policy is capable of at least attempting the strategies in the dataset. If the initial policy suffers from "mode collapse" (only learning one way to do things), the rollouts won't represent the full dataset, and the filter might fail.
- Success Indicators: The method requires an automated way to detect success/failure during rollouts. While often available in sim, this can be harder to implement in the real world without specialized sensors or secondary vision models.
- State Coverage: By filtering data, you naturally reduce the diversity of the training set. While this improves reliability for a specific task, it might theoretically reduce the robot's ability to recover from highly unusual out-of-distribution states.
- Future Direction: The authors suggest exploring how this curation could be integrated into "pre-training" large-scale foundation models for robotics.
Summary updated