TMA: Sim-to-Real Forceful Humanoid Manipulation via Tactile-Driven Motor Adaptation

Yiting Chen, Kelin Yu, Sarthak Kaingade, Bryan Trinh, Kaiyu Hang, Roberto Martin-martin, Nima Fazeli, X. Alice WuarXiv 2026 · arXiv

This paper, recently published by researchers from Amazon and several universities, introduces TMA (Tactile-Driven Motor Adaptation), a framework designed to help humanoid robots perform "forceful" manipulation tasks—specifically, opening round doorknobs—using zero-shot sim-to-real transfer.

TL;DR

TMA enables humanoid robots to open round doorknobs by adapting their motor behavior online based on a 2-second history of tactile feedback. By encoding contact forces using friction-cone geometry and training an adaptation module in simulation, the system achieves zero-shot transfer to a real-world Unitree G1 humanoid, significantly outperforming methods that rely solely on proprioception or reactive tactile sensing.

What Problem Does This Paper Address?

Most recent breakthroughs in humanoid "loco-manipulation" (moving while handling objects) are geometry-dominant. In these tasks, success is mostly about alignment: if the robot puts its hand in the right place and moves along a specific path, the door opens. Think of a lever handle—it’s a simple kinematic constraint.

However, interaction-dominant tasks, like turning a round doorknob, are much harder. Success depends on physical properties you can't see: How slippery is the knob? How stiff is the internal latch spring? Is the finger about to slip? Vision and joint positions (proprioception) often can't distinguish between a solid grasp and an imminent failure until it's too late. This "observability gap" requires a robot to sense and adapt to the evolving forces at its fingertips in real-time.

Methods

The authors propose a two-stage reinforcement learning (RL) pipeline called Tactile-Driven Motor Adaptation (TMA):

  1. Stage 1: The Oracle Policy. In simulation, a "privileged" encoder is trained alongside a manipulation policy. This encoder has access to "cheating" information—exact friction coefficients, latch stiffness, and door mass. It compresses this into a task-relevant latent representation that tells the policy how to behave.
  2. Stage 2: The Adaptation Encoder. The privileged information is taken away. Instead, a temporal-convolutional (TConv) encoder is trained to estimate that same latent representation using only a 2-second history of tactile and proprioceptive data.

Key Innovations: * Cone-Coordinate Representation: Instead of feeding raw XYZ force vectors into the neural network, the authors parameterize tactile signals into normal load, shear magnitude, and shear direction. This provides an "inductive bias" grounded in contact mechanics (the friction cone), making it easier for the policy to learn about slip and grip. * Whole-Body Integration: The system runs an upper-body manipulation policy (controlling 13 joints) in parallel with a frozen, pre-trained locomotion controller. This ensures the robot stays balanced even when it’s pulling or pushing hard on a door. * Interaction-Structured Rewards: The reward function is "gated" by tactile feedback. The robot only gets points for turning the knob if it is actually maintaining a proper multi-finger grasp, forcing the policy to prioritize stable, informative contact.

Datasets

The system was trained entirely in mjlab (a MuJoCo-based framework). To evaluate the "real" in sim-to-real, the authors tested the system on four different commercial round doorknobs with varying shapes:

  • Knob 1: Tapered barrel.
  • Knob 2: Spherical.
  • Knob 3: Axisymmetric disk.
  • Knob 4: Elongated olive shape.

This variety tests the robot's ability to generalize across different contact geometries and mechanical resistances without any fine-tuning in the real world.

Key Insights and Ideas

  • Tactile History is Mandatory: A "reactive" policy (looking only at the current frame) can tell if it's touching something, but it can't tell the difference between a stiff spring and a jammed latch. You need the temporal evolution of force to infer the underlying physics.
  • Proprioception is Insufficient for Stiff Hands: Standard Rapid Motor Adaptation (RMA) often uses joint errors to infer external forces. However, on non-backdrivable robot hands (like the BrainCo Revo2 used here), external forces don't cause much joint displacement. Tactile sensors are the only way to "see" the interaction.
  • Active Sensing: The policy learns to "probe" the environment. By attempting to turn the knob, it generates the tactile feedback necessary for the adaptation encoder to figure out the friction and stiffness of that specific door.

Key Results

  • Zero-Shot Success: The Unitree G1 humanoid successfully opened all four types of doorknobs in the real world.
  • TMA vs. RMA: TMA achieved high success rates (e.g., 8/9 trials for Knob 1), while the proprioception-based RMA baseline failed almost entirely, often drifting away from the knob because it couldn't sense the contact state accurately.
  • Ablation Study: Removing the shear (tangential) force information from the tactile signal significantly dropped performance, proving that sensing "twist" is just as important as sensing "squeeze" for this task.

Limitations and Future Work

  • Hardware Sensitivity: The authors noted that if the rubber fingertip pads were damaged, performance degraded. The policy relies on the friction characteristics it learned in simulation.
  • Task Scope: While successful for doorknobs, the framework hasn't yet been tested on other forceful tasks like using a screwdriver or opening a heavy jar.
  • Tactile Sim-to-Real Gap: While the cone-coordinate representation helps, simulating high-fidelity tactile sensors remains a challenge. Future work may involve more sophisticated sensor models or "tactile-to-tactile" translation layers.

Summary updated

Abstract

Recent humanoid loco-manipulation in articulated environments has largely focused on geometry-dominant interactions, where success is primarily determined by geometric alignment and motion execution. In contrast, forceful multi-finger manipulation requires the robot to adapt to real-world uncertainties in contact and interaction dynamics. We present Tactile-Driven Motor Adaptation (TMA) and an integrated whole-body system for zero-shot sim-to-real forceful humanoid manipulation, and study round-doorknob opening as a challenging benchmark requiring stable multi-finger frictional grasping, sufficient torque generation without slip, force-regulated pushing, and whole-body stability. TMA structures policy learning around interactions observable through tactile feedback, uses cone geometry as an inductive bias for multi-finger contact representation, and adapts motor behavior online from tactile history. An upper-body manipulation policy is trained alongside a pretrained general locomotion controller to coordinate dexterous interaction and whole-body stabilization. Trained entirely in simulation, TMA transfers zero-shot to a Unitree G1 humanoid with BrainCo Revo2 multi-fingered hands and substantially outperforms standard Rapid Motor Adaptation (RMA) and reactive tactile policy baselines across four commercial round doorknobs in the real world.
BibTeX
@misc{chen2026tma,
  author = {Yiting Chen and Kelin Yu and Sarthak Kaingade and Bryan Trinh and Kaiyu Hang and Roberto Martin-martin and Nima Fazeli and X. Alice Wu},
  title = {{TMA: Sim-to-Real Forceful Humanoid Manipulation via Tactile-Driven Motor Adaptation}},
  year = {2026},
  eprint = {2611.18976},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.18976},
}
Figure