WHED: A Wearable Hand Exoskeleton for Natural, High-Quality Demonstration Collection

Mingzhang Zhu, Alvin Zhu, Jose Victor S. H. Ramos, B Kim, Yike Shi, Yufeng Wu, Ruochen Hou, Quanyou Wang, Eric Song, Tony Fan, Yuchen Cui, Dennis Hong

TL;DR

This paper introduces WHED, a wearable hand exoskeleton designed to overcome the challenges of collecting natural, high-fidelity human demonstrations for robot manipulation. By prioritizing wearability and incorporating a novel "free-to-move" thumb mechanism, WHED enables extended, comfortable data collection while ensuring accurate mapping to robot hand movements.

What Problem Does This Paper Address?

Learning dexterous manipulation for robots is a significant challenge, largely due to the difficulty of acquiring high-quality human demonstration data. Existing methods struggle with issues like occlusion, complex hand kinematics, and the need for contact-rich interactions. While simulation and video data are abundant, they often lack the crucial physical interaction details. Robot teleoperation can provide direct control but is often slow, unintuitive, and limited by poor haptic feedback. Wearable interfaces, like gloves and exoskeletons, offer better embodiment but frequently compromise on comfort, wearability, and adaptability to different users, especially concerning the natural movement of the thumb. This gap hinders the development of more capable and human-like robotic manipulation skills.

Methods

The WHED system is built around two core principles: wearability-first and a pose-tolerant, free-to-move thumb.

  • Wearability-First Architecture: WHED emphasizes comfort for extended use. This includes a linkage-driven finger interface with passive fit accommodation, a modified passive hand with proprioceptive sensing, and an onboard sensing/power module for untethered operation. A passive gravity compensation system and a finger-ring support further enhance comfort and stability. Swappable thumbcots and holders allow for quick resizing to fit different users.
  • Pose-Tolerant, Free-to-Move Thumb: A key innovation is the thumb mechanism. It uses a multi-DOF interface that preserves the operator's natural thumb workspace, allowing it to translate and rotate relative to the palm while still transferring essential degrees of freedom to the passive hand. This is achieved through a linkage-slide mechanism that decouples the exoskeleton's motion from strict alignment with the passive hand's joints, ensuring comfort and adaptability.
  • Passive Hand Modification: The passive data-capture hand, based on the OYMotion ROH-AP001, was modified. A kinematic identification process using URDF simulations was performed to understand and decouple the underactuated thumb's internal joint dependencies. This informed the design of a custom linkage-slide mechanism for the exoskeleton, ensuring that recorded trajectories are kinematically feasible for the target robot.
  • Robust Sensing: High-fidelity proprioceptive data is captured using six joint encoders. To mitigate signal noise from sensor misalignment during dynamic manipulation, custom brackets with alignment ribs were engineered to rigidly mount the encoders, preventing backlash and detachment.
  • End-to-End Data Pipeline: The system includes a comprehensive pipeline for data collection and replay. This involves:
  • Data Collection: Capturing finger joint positions (1 kHz) via encoders, end-effector pose (60 Hz) using iPhone LiDAR with ARKit, and wrist-mounted visual observations (30 Hz) from a stereo camera.
  • Post-Processing: Synchronizing data streams using video timestamps as a master reference, filtering encoder and pose data to remove noise and outliers, and transforming the iPhone's coordinate frame to the end-effector's frame for accurate pose estimation.
  • Data Replay: Mapping synchronized and filtered human motion data to command the robotic platform, including calibrating finger encoders and applying a gain for responsive execution.

Datasets

The paper does not explicitly mention training datasets. The primary "dataset" generated is the high-fidelity demonstration data collected using the WHED system itself. This data consists of synchronized finger joint encoder readings, end-effector pose, and visual observations.

Key Insights and Ideas

  • The "Free-to-Move" Thumb is Crucial: The paper highlights that the thumb's complex, coupled movements are central to human dexterity. Forcing a rigid alignment for data collection sacrifices naturalness and comfort. WHED's pose-tolerant thumb mechanism is a significant step towards capturing these natural behaviors without compromising the ability to retarget them to a robot.
  • Wearability is Not a Trade-off for Fidelity: The authors demonstrate that a focus on wearability (comfort, adaptability, extended use) can coexist with high-fidelity data capture. This is achieved through clever mechanical design and passive accommodations.
  • End-to-End Pipeline is Essential: The success of collecting and utilizing this data relies heavily on a robust pipeline that handles synchronization, filtering, and transformation across multiple sensor modalities. This ensures that the collected data is not only accurate but also directly usable for robot control.
  • Passive Mechanisms Enhance Adaptability: The use of passive elements like the sliding finger cup and the gravity compensation system allows the exoskeleton to adapt to user anthropometrics and motion variations, improving comfort and reducing the burden on the operator.

Key Results

The paper demonstrates the feasibility of WHED through qualitative results and validation experiments.

  • Representative Tasks: The system was used to collect demonstrations for grasping and manipulation sequences, including precision pinch grasps (e.g., USB stick) and full-hand enclosure grasps (e.g., water bottle).
  • Qualitative Consistency: Replaying collected demonstrations on a robot showed a high degree of kinematic similarity in both spatial translation and rotation of the end-effector. The robot achieved stable grasps with orientations matching the operator's, and objects were released at nearly identical locations.
  • Reproducibility: The data replay validation confirmed that the collected data is high-quality and reproducible, with minimal deviation from the original human demonstrations.
  • Wearability: While not quantified with specific metrics, the design choices (passive elements, swappable parts, gravity compensation) strongly suggest improved wearability for extended sessions.

Limitations and Future Work

  • Limitations:
    • The paper acknowledges that detailed quantitative kinematic modeling, workspace/singularity analysis, and controlled user studies will be reported in a follow-up submission.
  • While qualitative results are strong, specific quantitative metrics for data accuracy (e.g., retargeting error) are not detailed in this paper.
  • The current system focuses on hand manipulation; integrating arm control more deeply might be a future direction.
  • Future Work:
    • The authors plan to report detailed quantitative analyses and user studies in a future publication.
  • The end-to-end pipeline is designed for future extensions and improvements.
  • The system's ability to capture complex, contact-rich tasks with minimal deviation suggests potential for learning more sophisticated manipulation skills.

Summary updated

Abstract

Scalable learning of dexterous manipulation remains bottlenecked by the difficulty of collecting natural, high-fidelity human demonstrations of multi-finger hands due to occlusion, complex hand kinematics, and contact-rich interactions. We present WHED, a wearable hand-exoskeleton system designed for in-the-wild demonstration capture, guided by two principles: wearability-first operation for extended use and a pose-tolerant, free-to-move thumb coupling that preserves natural thumb behaviors while maintaining a consistent mapping to the target robot thumb degrees of freedom. WHED integrates a linkage-driven finger interface with passive fit accommodation, a modified passive hand with robust proprioceptive sensing, and an onboard sensing/power module. We also provide an end-to-end data pipeline that synchronizes joint encoders, AR-based end-effector pose, and wrist-mounted visual observations, and supports post-processing for time alignment and replay. We demonstrate feasibility on representative grasping and manipulation sequences spanning precision pinch and full-hand enclosure grasps, and show qualitative consistency between collected demonstrations and replayed executions.
BibTeX
@misc{zhu2026whed,
  author = {Mingzhang Zhu and Alvin Zhu and Jose Victor S. H. Ramos and B Kim and Yike Shi and Yufeng Wu and Ruochen Hou and Quanyou Wang and Eric Song and Tony Fan and Yuchen Cui and Dennis Hong},
  title = {{WHED: A Wearable Hand Exoskeleton for Natural, High-Quality Demonstration Collection}},
  year = {2026},
  eprint = {2602.17908},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2602.17908},
}
Figure