AsymEEF: Bimanual Manipulation with Asymmetric End Effectors

Xiaokang Sun, Ningyue Wang, Kunqi Xu, Guangqi Jiang, Jianglong Ye, Yong Wu, Xueyan ZouarXiv 2026 · arXiv

TL;DR

AsymEEF is a framework that allows bimanual robots to "reuse" a small set of teleoperated demonstrations across different combinations of end-effectors (e.g., transferring a task from two grippers to a gripper and a five-fingered hand). By mapping diverse hardware into a shared, morphology-agnostic latent space and using physics-based replay to filter generated data, it achieves a 16.6× data expansion in simulation and enables training a single, unified policy for heterogeneous robot pairs.

What Problem Does This Paper Address?

In the world of imitation learning, teleoperation is the gold standard for data collection but is notoriously expensive and time-consuming. This "data tax" becomes a massive burden for bimanual manipulation. If you have $N$ different types of robotic hands (grippers, three-fingered hands, dexterous five-fingered hands), there are $N^2$ possible bimanual combinations.

Currently, if you collect data for a robot with two grippers, that data is largely useless if you swap one gripper for a dexterous hand because the kinematics and action spaces are completely different. AsymEEF tackles this "asymmetry" problem, aiming to generate executable demonstrations for any target hand pair from just a few "seed" demonstrations, without requiring new manual teleoperation for every hardware configuration.

Methods

The authors propose a multi-stage pipeline to bridge the gap between different robotic morphologies:

  1. Shared EEF-State Interface: The core is a Transformer-based encoder-decoder architecture. The encoder takes the wrist pose and actuator states (regardless of the number of fingers) and maps them into a fixed 24-dimensional morphology-agnostic latent space. A morphology-conditioned decoder can then "translate" this latent intent back into specific joint commands for a different target hand.
  2. Cross-EEF Anchoring: To train this shared space, the authors use two types of "anchors":
  3. Geometry-Guided Teacher Anchors: These use differentiable forward kinematics to align functional points (like grasp centers, opening widths, and approach directions) between different hands.
  4. X-Calib (Real-World Calibration): Since pure kinematics can't account for physical offsets or contact compliance, the authors collect a small set of manual "keyframe" poses in the real world to fine-tune the mapping.
  5. Progressive Spatial Expansion & Replay Filtering: Once a demonstration is transferred to a new hand pair, the system perturbs the trajectory to cover more of the workspace. Crucially, every generated trajectory is replayed in a physics simulator (or on the real robot). Only those that successfully complete the task and satisfy collision/kinematic constraints are kept.
  6. Unified Policy Learning: Because the representation is hardware-agnostic, the authors can train a single visuomotor policy (using backbones like ACT or $\pi_{0.5}$) that works across multiple different hand combinations simultaneously.

Datasets

The researchers evaluated AsymEEF across both simulation and real-world environments:

  • Simulation (SAPIEN): Four complex tasks including Can Sorting, Handover, Lift Tray, and Laptop & Mouse.
  • Real-World: Six tasks involving an xArm7 setup with XHANDs (dexterous) and G2 grippers. Tasks included Toy Sorting, Motherboard (precise insertion), CPU Installation, and Screwdriver Bit removal.
  • X-Calib Set: A small, newly introduced calibration set of 216 frames collected in under an hour to align the XHAND and G2 gripper.

Key Insights and Ideas

  • Intent over Mechanics: The "aha" moment is that while a gripper and a human-like hand move differently, their functional intent (where they are grasping and from what direction) is the same. By encoding this intent rather than raw joint angles, the system achieves cross-hardware portability.
  • Replay as a Quality Gate: Instead of trying to make the "translation" perfect, AsymEEF uses the physics engine as a filter. It generates many candidates and lets the environment decide which ones are valid demonstrations.
  • Asymmetry as a Training Signal: Training a single policy on heterogeneous pairs (e.g., Left=Hand, Right=Gripper) actually helps the model learn more robust coordination than training on symmetric pairs alone.

Key Results

  • Data Expansion: In simulation, AsymEEF achieved a mean 16.6× expansion gain. This means for every 1 teleoperated demo, the system generated over 16 unique, valid demonstrations for different hand pairs.
  • Superior Transfer: AsymEEF significantly outperformed existing baselines (LAD and CEI) in "Replay Success Rate." For example, on the real-world Toy Sorting task, AsymEEF achieved a 0.94 success rate compared to 0.29 for LAD.
  • High-Precision Success: On the CPU Installation task—which requires sub-centimeter precision—AsymEEF reached a 79% transfer success rate, while baseline methods often failed entirely (0%).
  • Unified Policy Performance: The downstream bimanual policies trained on this generated data achieved an average success rate of 83.3% across all real-world tasks.

Limitations and Future Work

  • Tabletop Constraint: The current evaluation is focused on tabletop manipulation. Expanding this to mobile manipulation or more dynamic environments remains a challenge.
  • Manual Calibration: While the X-Calib process is brief (under an hour), it still requires manual effort. The authors suggest that automated self-calibration could be a promising future direction.
  • Task Complexity: While the tasks are contact-rich, they are still relatively short-horizon. Testing the framework on long-chain multi-stage tasks would further prove its scalability.

Summary updated

Abstract

Teleoperated demonstrations are costly to collect and rarely reusable across end-effectors with different kinematics and action spaces. This burden grows in bimanual manipulation, where each end-effector pair may require new data. We present AsymEEF, a framework for executable cross-end-effector demonstration generation and unified bimanual policy learning. Given a small source dataset, AsymEEF transfers demonstrations to heterogeneous combinations of two-, three-, and five-fingered end-effectors without additional teleoperation. Kinematic teacher anchors provide geometric supervision, while sparse real-world X-Calib anchors correct discrepancies in grasp execution. A shared encoder maps wrist poses and actuator states to a morphology-agnostic latent space, and a morphology-conditioned decoder reconstructs target commands. Generated trajectories undergo replay filtering for task-level executability. The representation also supports a unified bimanual visuomotor policy trained across multiple pairs. Across four simulated and six real-world tasks, AsymEEF improves replay success over adapted transfer baselines, achieves a mean replay-valid expansion gain of 16.6 times in simulation, and is evaluated through downstream policy training on real robots. Project website: https://asymeef.vercel.app/.
BibTeX
@misc{sun2026asymeef,
  author = {Xiaokang Sun and Ningyue Wang and Kunqi Xu and Guangqi Jiang and Jianglong Ye and Yong Wu and Xueyan Zou},
  title = {{AsymEEF: Bimanual Manipulation with Asymmetric End Effectors}},
  year = {2026},
  eprint = {2611.20557},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.20557},
}
Figure