A Head Camera Is (Almost) All You Need: An Electronics-Free UMI Glove Labelled and Observed by One Moving Camera

Yosub Shin, Yuan Liu, Igor MolybogarXiv 2026 · arXiv

TL;DR

This paper introduces a low-cost ($144), electronics-free system for collecting robot demonstration data using a head-mounted camera and a passive, 3D-printed glove. By mimicking human gaze during deployment, the system outperforms traditional wrist-camera setups and significantly boosts data efficiency, making high-quality imitation learning more accessible.

What Problem Does This Paper Address?

Scaling up robot learning requires massive amounts of demonstration data. While handheld devices allow users to collect data without tying up a physical robot, current solutions (like the Universal Manipulation Interface or UMI) are often cumbersome. They typically require wrist-mounted cameras, IMUs, or VR controllers, leading to heavy hardware, tangled cables, and complex multi-stream synchronization issues.

Furthermore, there is a "viewpoint gap." A human collector naturally looks at the task, but a robot's wrist camera might lose sight of the goal (like a bin) during a carry motion. This paper asks: Can we simplify the hardware to a single camera and a passive glove while actually improving robot performance?

Methods

The authors propose a system centered around a single egocentric viewpoint.

  1. Hardware: The collector wears a standard 4K camera on a head strap ($122) and holds an electronics-free, 3D-printed replica of the robot's gripper ($22).
  2. Tracking: The glove is covered in fiducial markers—an ArUco dodecahedron on the back and AprilTag "tails" on the jaws. By tracking these against paper tiles taped to the workspace, the system extracts the 6-DoF gripper pose and jaw width directly from the video frames.
  3. Deployment: To bridge the human-to-robot gap, the same camera is mounted on a low-cost, 6-joint servo arm (a "positioner") next to the robot. This positioner uses a regression model to mimic the collector's head movements, ensuring the robot sees the workspace exactly as the human did during data collection.
The system overview: (a) human data collection, (b) the camera's view with fiducials, and (c) robot deployment with the camera positioner.

Datasets

The study utilized two primary data sources for training an Action Chunking with Transformer (ACT) policy:

  • Teleoperation Data: 178 episodes (approx. 38.6k frames) recorded using the robot positioner.
  • Glove Data: 198 demonstrations (approx. 50.0k frames) collected by a human wearing the head camera and passive glove.

Key Insights and Ideas

  • The Power of Perspective: The head-mounted viewpoint is superior to a wrist-mounted one because it naturally keeps both the gripper and the target (e.g., a bin) in the same frame. Wrist cameras often "look away" during transport, leading to failures where the robot drops the object.
  • Data Multiplier: Glove demonstrations are highly efficient. The authors found that adding ~200 glove demos to a small set of 48 teleoperated episodes boosted success rates to the same level as having 172 teleoperated episodes alone.
  • Passive is Robust: By eliminating electronics from the glove, the authors removed the need for battery management, wireless lag, and complex sensor fusion. A single video stream provides both the visual observation and the ground-truth labels.
Figure A1: Robot validation loss against the number of teleoperated episodes, with and without the 194 glove segments. Solid: head-camera-labelled teleoperation (round-2 chain, four budgets); dashed: encoder-labelled teleoperation (final design, two budgets). The glove set moves every bud- get to the level of 2–4× as many teleoperated episodes.

Key Results

  • Accuracy: The system tracks the tool point to within 6.5 mm and 1.55° median error compared to robot encoders.
  • Success Rates: The full co-trained head-camera policy achieved a 20/25 success rate, significantly outperforming the UMI-style wrist-camera recipe (15/25) and a fixed-camera setup (12/25).
  • Failure Analysis: Wrist-camera policies failed primarily by dropping objects mid-carry when the bin left their field of view. The head-camera policies remained robust because the "gaze" followed the action.
Figure A3: One placement (scene 16), the four camera arms, at 2, 6, 10 and 14 s, from the fixed workspace camera with each policy’s own view inset. During the carry the wrist view (rows 2 and 4) holds the block and the far wall and nothing else; the head view (row 1) keeps gripper and bin in one frame. Arm 8 opened with the bin out of its view.

Limitations and Future Work

While promising, the system has a few hurdles:

  • Positioner Lag: The robot-side camera arm currently has a 0.4s lag, which could affect high-speed tasks.
  • Occlusion: The robot's own hardware sometimes occludes the markers, though the authors suggest this can be mitigated by removing unnecessary brackets.
  • Generalization: The current model was tested on a single task with a single collector. Future work aims to explore bimanual manipulation, multi-tile workspaces, and "in-the-wild" data collection across diverse environments.

Summary updated

Abstract

Handheld demonstration devices decouple data collection from the robot, but conventional designs burden the hand with a wrist camera, cables and a pose tracker (SLAM, an IMU, a VR controller, or a fixed external camera that the collector's body occludes), and burden the pipeline with as many data streams to timestamp, synchronise and cross-calibrate. We show that a single egocentric camera (USD 122) worn on the collector's head replaces the wrist camera, the pose tracker and their separate streams with one. It reads printed fiducials on an electronics-free glove (USD 22) against paper tiles taped around the workspace, giving the 6-DoF gripper pose and the jaw width from the same frames the policy will see; at deployment the same camera rides a low-cost (USD 120) arm beside the robot that mimics the collector's head from the robot's joint state. Against the robot's encoders the labels are accurate to 6.5 mm and 1.6 degrees median with 93-100% frame coverage, and a policy trained on them instead of encoders pays 3% in validation loss. Adding 194 glove demonstrations to 48 teleoperated episodes lifts rollout success from 10/25 to 17/25, level with 172 teleoperated episodes alone (18/25), and the full co-trained head-camera policy succeeds in 20/25 against 15/25 for the wrist-camera recipe, whose policies drop the object when the goal leaves their view, and 12/25 for a fixed camera.
BibTeX
@misc{shin2026head,
  author = {Yosub Shin and Yuan Liu and Igor Molybog},
  title = {{A Head Camera Is (Almost) All You Need: An Electronics-Free UMI Glove Labelled and Observed by One Moving Camera}},
  year = {2026},
  eprint = {2610.27622},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.27622},
}
Figure