Abstract
While humans can use parts of their arms other than the hands for manipulations like gathering and supporting, whether robots can effectively learn and perform the same type of operations remains relatively unexplored. As these manipulations require joint-level control to regulate the complete poses of the robots, we develop AirExo, a low-cost, adaptable, and portable dual-arm exoskeleton, for teleoperation and demonstration collection. As collecting teleoperated data is expensive and time-consuming, we further leverage AirExo to collect cheap in-the-wild demonstrations at scale. Under our in-the-wild learning framework, we show that with only 3 minutes of the teleoperated demonstrations, augmented by diverse and extensive in-the-wild data collected by AirExo, robots can learn a policy that is comparable to or even better than one learned from teleoperated demonstrations lasting over 20 minutes. Experiments demonstrate that our approach enables the model to learn a more general and robust policy across the various stages of the task, enhancing the success rates in task completion even with the presence of disturbances.
Introduction
In robotic manipulation research, the primary focus has historically been on end-effectors—the "fingertips" of the robot. However, human manipulation is inherently "whole-arm." We use our elbows to nudge doors shut, our forearms to sweep objects together, and our upper arms to support heavy loads. Replicating this in robotics is challenging because it requires precise joint-level control and carries a high risk of unintended collisions.
The standard approach to teaching these skills is teleoperation, where a human expert controls the robot directly. While effective, teleoperation is slow, expensive, and requires the robot to be physically present. Conversely, "in-the-wild" human video data is abundant but suffers from the "correspondence problem": the kinematic differences between a human arm and a robotic arm make it difficult for an AI to map human movements directly to robot motor commands.
The AirExo System
To bridge this gap, we introduce AirExo, a portable, low-cost dual-arm exoskeleton designed for both teleoperation and in-the-wild data collection.
Hardware Design
AirExo is designed with accessibility in mind. Built almost entirely from 3D-printed PLA plastic and standard off-the-shelf encoders, the hardware costs approximately $600. Its mechanical design is "universal," featuring adjustable linkages that can be modified to match the kinematics of various industrial and research robots, such as the UR5, Franka, or Kuka. This ensures that the data collected by a human wearer is immediately relevant to the target robot's configuration.
Learning Framework
We propose a two-stage imitation learning schema that leverages the strengths of both in-the-wild and teleoperated data:
- Stage 1: In-the-Wild Pre-training: A human wearer performs tasks in a natural environment (e.g., a kitchen or office) without a robot present. The exoskeleton’s encoders record joint angles directly. Because the device mimics the robot's kinematics, this data is already in the robot's "language," bypassing the need for complex pose estimation from video.
- Stage 2: Teleoperation Fine-tuning: The human uses AirExo to teleoperate the actual robot for a brief period. This stage allows the model to learn fine-grained details of robot-specific physics, such as friction and contact dynamics.
The underlying architecture utilizes Action Chunking with Transformers (ACT), which is particularly effective at handling the multi-modal nature of complex, contact-rich tasks.
Experimental Evaluation
We evaluated the AirExo framework on two challenging whole-arm manipulation tasks:
- Gather Balls: A contact-rich task requiring the robot to use its forearms to sweep 80 cotton balls into a designated area.
- Grasp from Curtained Shelf: A multi-stage task where one arm holds back a transparent curtain while the other reaches in to retrieve a toy.
Key Results
Our experiments revealed significant improvements in sample efficiency and robustness:
- Sample Efficiency: The most striking finding was that just 3 minutes of teleoperation (10 demonstrations) augmented with in-the-wild data outperformed 20 minutes (50 demonstrations) of teleoperation alone.
- Success Rates: In the curtained shelf task, the AirExo framework achieved an 88% success rate with only 10 teleop demonstrations. Without in-the-wild pre-training, the success rate for the same amount of data dropped to 44%.
- Robustness and Generalization: Policies trained with in-the-wild data were significantly more resilient to disturbances. When tested with novel objects (toys not seen during training), the AirExo framework maintained a 7/8 success rate, whereas teleoperation-only baselines dropped to 4/8. Furthermore, the models maintained a 0% collision rate in most scenarios, despite the high-contact nature of the tasks.
Discussion and Future Work
AirExo demonstrates that low-cost hardware can solve high-level kinematic correspondence problems. By recording joint angles directly in the wild, we provide the robot with a diverse dataset that prevents the "laziness" or arm-imbalance often seen in small teleoperated datasets.
While successful, a visual domain gap remains; the perspective of a human in a kitchen differs from a robot in a lab. Future work will focus on improving image processing to bridge this visual gap and moving toward a "robot-free" learning goal, where robots can learn complex manipulations entirely from in-the-wild human data without any teleoperation required.
Summary updated