ActiveAlign: Aligning Egocentric Manipulation via Active Perception

Ri-Zhao Qiu, Bowen Wen, Yifeng Jiang, Chen Tessler, Eric Heiden, Xue Bin Peng, Xiaolong Wang, Stan Birchfield, Erwin CoumansarXiv 2026 · arXiv

TL;DR

ActiveAlign is a new framework that uses active perception—human-like motions like hand-waving and tactile probing—to bridge the gap between simulation and reality. By actively calibrating cameras and refining object positions through touch, it enables humanoid robots to perform high-precision tasks like pressing small buttons and moving sliders with millimeter-level accuracy.

What Problem Does This Paper Address?

In the field of robotics, "Real-to-Sim-to-Real" (Real2Sim2Real) is a popular paradigm where we build a digital twin of the world to plan and train robot behaviors. However, this approach often fails during fine manipulation. Why? Because of the Real2Sim gap.

Standard methods rely on "passive perception"—taking a static picture and trying to guess where everything is. Even tiny errors in camera calibration or object tracking get amplified through a robot's long kinematic chain (the "compounding error" problem). For a humanoid robot, a 5mm error in perception can mean the difference between successfully flipping a switch and poking the wall next to it.

Methods

ActiveAlign moves away from passive observation, drawing inspiration from how humans move their heads to see better or feel around for a light switch in the dark. It introduces two main modules:

  1. ActiveAlign-Waving (Visual Calibration): To calibrate egocentric (head-mounted) cameras, the robot "waves" its hands. The system uses differentiable rendering—specifically 3D Gaussian Splatting (3DGS) combined with Linear Blend Skinning—to compare what the robot should see (based on its internal model) with what it actually sees. By optimizing this difference over a sequence of motions, it achieves much more stable camera calibration than single-frame methods.
  2. ActiveAlign-Touching (Tactile Refinement): Even with perfect cameras, motor backlash and joint errors persist. The robot uses a tactile-sensing finger to "feel" the target area, building a local height map. This physical feedback allows the robot to align its actions to the actual physical surface of an object, compensating for any remaining millimeter-scale errors.
The ActiveAlign pipeline consists of an offline stage for asset creation and an online stage where active perception updates the simulation model.

Datasets

The authors evaluated ActiveAlign-Waving on several open-sourced robot manipulation datasets to prove its generalizability:

  • Mobile ALOHA: A popular mobile bimanual platform.
  • Humanoid Everyday (Unitree G1): A dataset featuring a modern humanoid robot.
  • xdof ZED-X: A stationary robot setup.
  • ABC-130K: Used for qualitative cross-dataset validation.

For the physical experiments, they used a Unitree G1 humanoid equipped with a tactile-sensing "RoboSkin" on its index finger.

Key Insights and Ideas

  • Active > Passive: The "aha moment" here is that robot motion shouldn't just be the result of perception; it should be a tool for perception. Waving hands provides diverse viewpoints that resolve the ambiguities (degenerate solutions) common in egocentric vision.
  • Importance Sampling for Humanoids: Humanoids have high degrees of freedom (DoFs). The authors found that standard calibration fails because the camera often only sees a small part of the arm. By using importance sampling to focus the optimization on the end-effectors, they significantly improved calibration stability.
  • Closing the Loop with Touch: Visual alignment gets you close, but tactile alignment gets you to the finish line. The height map approach (Figure 3) effectively "zeros out" the cumulative errors from the entire vision-to-actuation pipeline.
Tactile sensing allows the robot to build a height map, correcting for the gap between where it thinks the object is and where it actually is.

Key Results

ActiveAlign showed dramatic improvements in both calibration and task success:

  • Calibration Accuracy: On the Unitree G1, ActiveAlign achieved an Intersection over Union (IoU) of 0.608 for robot mask alignment, nearly doubling the performance of standard URDF-based projection (0.361).
  • Fine Manipulation Success: In tasks requiring millimeter precision, the difference was stark. For a "Midi Slider" task, a standard keypoint planning baseline succeeded only 3/20 times, while ActiveAlign + Motion Primitives succeeded 14/20 times. Similar jumps were seen for toggle switches and light sliders.
  • Generalization: The system successfully calibrated public datasets (like Mobile ALOHA) that were originally released without camera calibration parameters, enabling them to be used in high-fidelity simulations.
Qualitative results showing ActiveAlign-Waving calibrating various public datasets, aligning the simulated robot (colored) with the real robot (grayscale).

Limitations and Future Work

While impressive, ActiveAlign has a few areas for growth:

  • Tactile Scope: The current tactile alignment assumes a roughly 2D planar surface to build its height map. Extending this to complex 3D geometries would be a natural next step.
  • Speed: The tactile probing process (touching the surface to build a map) takes time. Future work could explore more efficient "active exploration" strategies to minimize this overhead.
  • Offline Dependency: The system still requires an offline stage to build the initial 3DGS model of the robot, which might be a hurdle for rapid deployment on entirely new robot designs.

Summary updated

Abstract

Real-to-Sim-to-Real (Real2Sim2Real) enables structured world representations for planning, interpretability, and safe execution. Most existing approaches rely on passive perception, which creates real2sim gaps due to imperfect observation. Inspired by how humans perceive actively through actions, we propose ActiveAlign, a framework that leverages active perception to align the Real2Sim2Real gap from interactions. ActiveAlign introduces two complementary modules: (i) ActiveAlign-Waving, which performs hand-eye calibration by exploiting active hand motions and differentiable rendering, and (ii) ActiveAlign-Touching, which refines object alignment through tactile feedback to achieve millimeter-level precision. Together, these capabilities enable strong real2sim alignment. We demonstrate that ActiveAlign-Waving can be used to curate simulation environments for open-sourced uncalibrated robot manipulation datasets, and demonstrate on a real Unitree G1 robot that ActiveAlign enables fine manipulation of millimeter-scale objects such as pressing buttons and moving sliders.
BibTeX
@misc{qiu2026activealign,
  author = {Ri-Zhao Qiu and Bowen Wen and Yifeng Jiang and Chen Tessler and Eric Heiden and Xue Bin Peng and Xiaolong Wang and Stan Birchfield and Erwin Coumans},
  title = {{ActiveAlign: Aligning Egocentric Manipulation via Active Perception}},
  year = {2026},
  eprint = {2610.36859},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.36859},
}
Figure