TL;DR
This paper introduces a low-cost ($144), electronics-free system for collecting robot demonstration data using a head-mounted camera and a passive, 3D-printed glove. By mimicking human gaze during deployment, the system outperforms traditional wrist-camera setups and significantly boosts data efficiency, making high-quality imitation learning more accessible.
What Problem Does This Paper Address?
Scaling up robot learning requires massive amounts of demonstration data. While handheld devices allow users to collect data without tying up a physical robot, current solutions (like the Universal Manipulation Interface or UMI) are often cumbersome. They typically require wrist-mounted cameras, IMUs, or VR controllers, leading to heavy hardware, tangled cables, and complex multi-stream synchronization issues.
Furthermore, there is a "viewpoint gap." A human collector naturally looks at the task, but a robot's wrist camera might lose sight of the goal (like a bin) during a carry motion. This paper asks: Can we simplify the hardware to a single camera and a passive glove while actually improving robot performance?
Methods
The authors propose a system centered around a single egocentric viewpoint.
- Hardware: The collector wears a standard 4K camera on a head strap ($122) and holds an electronics-free, 3D-printed replica of the robot's gripper ($22).
- Tracking: The glove is covered in fiducial markers—an ArUco dodecahedron on the back and AprilTag "tails" on the jaws. By tracking these against paper tiles taped to the workspace, the system extracts the 6-DoF gripper pose and jaw width directly from the video frames.
- Deployment: To bridge the human-to-robot gap, the same camera is mounted on a low-cost, 6-joint servo arm (a "positioner") next to the robot. This positioner uses a regression model to mimic the collector's head movements, ensuring the robot sees the workspace exactly as the human did during data collection.
Datasets
The study utilized two primary data sources for training an Action Chunking with Transformer (ACT) policy:
- Teleoperation Data: 178 episodes (approx. 38.6k frames) recorded using the robot positioner.
- Glove Data: 198 demonstrations (approx. 50.0k frames) collected by a human wearing the head camera and passive glove.
Key Insights and Ideas
- The Power of Perspective: The head-mounted viewpoint is superior to a wrist-mounted one because it naturally keeps both the gripper and the target (e.g., a bin) in the same frame. Wrist cameras often "look away" during transport, leading to failures where the robot drops the object.
- Data Multiplier: Glove demonstrations are highly efficient. The authors found that adding ~200 glove demos to a small set of 48 teleoperated episodes boosted success rates to the same level as having 172 teleoperated episodes alone.
- Passive is Robust: By eliminating electronics from the glove, the authors removed the need for battery management, wireless lag, and complex sensor fusion. A single video stream provides both the visual observation and the ground-truth labels.
Key Results
- Accuracy: The system tracks the tool point to within 6.5 mm and 1.55° median error compared to robot encoders.
- Success Rates: The full co-trained head-camera policy achieved a 20/25 success rate, significantly outperforming the UMI-style wrist-camera recipe (15/25) and a fixed-camera setup (12/25).
- Failure Analysis: Wrist-camera policies failed primarily by dropping objects mid-carry when the bin left their field of view. The head-camera policies remained robust because the "gaze" followed the action.
Limitations and Future Work
While promising, the system has a few hurdles:
- Positioner Lag: The robot-side camera arm currently has a 0.4s lag, which could affect high-speed tasks.
- Occlusion: The robot's own hardware sometimes occludes the markers, though the authors suggest this can be mitigated by removing unnecessary brackets.
- Generalization: The current model was tested on a single task with a single collector. Future work aims to explore bimanual manipulation, multi-tile workspaces, and "in-the-wild" data collection across diverse environments.
Summary updated