TL;DR
CrowdMAPS is a crowdsourcing framework that breaks the "one user, one robot" bottleneck in imitation learning by allowing remote annotators to provide multiple valid alternative actions for already-visited robot states. By densifying supervision at keyframes through a web-based 3D interface, it boosts policy success rates by over 30 percentage points without requiring additional physical robot time.
What Problem Does This Paper Address?
Imitation learning (IL) for robotics is notoriously data-hungry and expensive. Unlike computer vision or natural language processing, where data can be scraped from the web or labeled asynchronously, robot data collection is usually hardware-bound. Standard teleoperation requires a "one user, one robot" synchronous model: to get one more action label, you need a human to control a physical robot in real-time.
Furthermore, standard datasets typically record only a single action for any given state—the one the expert happened to take. This ignores the inherent multimodality of many tasks (e.g., there are many ways to grasp a cup or different orders to sort objects). This "one state, one action" limitation makes it difficult for robots to learn robust, generalized policies from sparse demonstrations.
Methods
The authors propose CrowdMAPS, a paradigm that shifts the focus from collecting more trajectories ("length") to collecting more actions per state ("width").
The workflow consists of two main stages:
- State Acquisition: An expert collects a small number of standard real-robot trajectories.
- Action Densification: Visited keyframes are distributed to remote, non-expert annotators. Using a "Real2Sim" pipeline—which uses LangSAM for segmentation and Any6D for pose estimation to reconstruct the scene in Isaac Lab—annotators use a web interface to specify alternative valid goal poses ($SE(3)$ coordinates plus gripper state).
To train policies on this data, the authors use Action-Type Weighting. This gives the highest weight to physically executed actions and lower weights to unexecuted alternatives and intermediate frames, helping to mitigate "MAPS shift"—the distributional mismatch caused by having actions in the training set that don't have corresponding successor states.
Datasets
The researchers evaluated CrowdMAPS on five real-world tabletop manipulation tasks:
- Drawer: Opening a drawer and placing a cube inside.
- Insertion: Precisely inserting a pole into a cylinder.
- Pouring: Picking up a cup of nails and pouring them into a container.
- Sorting: Placing colored cubes into matching containers (multimodal).
- Switches: Pressing a sequence of puck lights (multimodal).
They recruited 47 participants (mostly non-experts) who provided thousands of annotations. Median submission times for these annotations ranged from just 7 to 12 seconds.
Key Insights and Ideas
- Asynchronous Scaling: By decoupling state acquisition from action labeling, CrowdMAPS allows an arbitrary number of people to contribute to a single robot's learning process simultaneously, bypassing the physical execution bottleneck.
- Variance Reduction: The authors provide a theoretical proof showing that Multiple Actions per State (MAPS) strictly reduces the variance of the imitation learning gradient estimator compared to single-action datasets.
- Sharpening and Separation: For unimodal tasks (like Insertion), increasing the "width" of the dataset reduces the dispersion of policy predictions. For multimodal tasks (like Sorting), it helps the policy clearly separate distinct valid modes rather than collapsing into a "blurry" average.
Key Results
- Massive Success Rate Boost: For keypose policies, increasing the dataset width from 1 to 10 actions per state improved average success rates from 38.7% to 74.2% (a 35.6 percentage point gain).
- Outperforming Expert Demos: In trajectory-based diffusion policies, adding MAPS data was more effective than adding an equal or greater number of full expert-teleoperated trajectories.
- Sample Efficiency: The gains were most pronounced in longer-horizon and multimodal tasks, where traditional single-action demonstrations struggle to cover the space of valid behaviors.
Limitations and Future Work
- Task Complexity: The current implementation focuses on quasi-static, single-arm tasks. Extending this to dynamic, contact-rich, or bimanual tasks may require more sophisticated annotation interfaces (e.g., short motion segments instead of just goal poses).
- Real-to-Sim Gap: The pipeline currently requires manual CAD models for scene reconstruction. Moving toward more generalizable, model-free reconstruction would make the system easier to deploy for new tasks.
- Human Validation: While the annotation is crowdsourced, a researcher still manually validates submissions for safety and strategy. Automating this validation via learned models or consensus-based filtering is a key next step.
- Successor States: Because alternative actions are never physically executed, the robot doesn't know what the world looks like after those actions. Future work could explore using world models to synthesize these missing successor states.
Summary updated