TL;DR
TacCoRL introduces a scalable framework to inject tactile feedback into pretrained Vision-Language-Action (VLA) models using a real-aligned simulator. By combining sim-real co-training with reinforcement learning anchored by real-world data, it enables robots to learn fine-grained contact corrections for complex bimanual tasks without requiring massive real-world exploration or large-scale tactile pretraining.
What Problem Does This Paper Address?
Pretrained Vision-Language-Action (VLA) models excel at leveraging massive visual and language datasets to guide robot manipulation. However, vision alone falls short in contact-rich manipulation tasks—such as inserting a test tube into a tight rack or assembling interlocking parts. In these scenarios, critical contact regions are frequently occluded by the robot's gripper or the object itself.
While tactile sensing provides the necessary local feedback to resolve these ambiguities, integrating touch into VLAs is notoriously difficult. Large-scale tactile datasets do not exist on the scale of internet vision data, and collecting near-failure recovery data directly on real hardware risks damaging expensive sensors and robots.
Methods
TacCoRL bridges this gap using a two-stage sim-to-real post-training pipeline:
- Tactile-Augmented VLA Architecture: Built on top of a $\pi_{0.5}$ VLA backbone, TacCoRL processes a history window of tactile readings through a CNN encoder to produce tactile tokens. A binary contact gate filters out background noise, ensuring tactile tokens only influence the model when actual contact occurs, thereby preserving the VLA's free-space visual priors.
- Simulation Alignment (Digital Twin): To train safely, the authors build a high-fidelity simulator aligned with the real world. They use System Identification (SysID) via SPSA optimization to match the real robot's controller response and calibrate a spring-damper tactile model to align simulated contact readings with real ones.
- Stage I: Sim-Real Co-training: The model is first fine-tuned via supervised flow-matching on a mixed dataset of real and simulated expert demonstrations. This warm-starts the policy with an initial tactile-conditioned action prior.
- Stage II: Tactile-Guided RL with a Real-Data Anchor: The policy is optimized using Proximal Policy Optimization (PPO) in simulation based on sparse task rewards. Crucially, a supervised real-data behavior cloning term acts as an "anchor," preventing the policy from exploiting simulator physics and drifting away from the real-world data distribution.
Datasets
The framework was evaluated on four custom bimanual contact-rich tasks: Test Tube Insertion, Do Puzzle, Assembly #1, and Assembly #2.
The dataset composition per task includes:
- 50 real-world human demonstrations collected via teleoperation.
- 20 simulated teleoperation trajectories.
- 200 synthesized trajectories generated in simulation using MimicGen to bootstrap the reinforcement learning exploration process.
Key Insights and Ideas
- Don't Just Add Touch—Learn to React: The core value of tactile feedback isn't just knowing contact happened, but knowing how to modulate actions in near-failure states. Simulation provides a safe sandbox to experience these rare, off-nominal states that are missing from standard human demonstrations.
- The Power of the Real-Data Anchor: Pure simulation RL often leads to sim-to-real gap failures because the policy overfits to simulator quirks. Adding a real-data anchor loss during RL fine-tuning constrains the policy update, keeping it grounded in real-world deployment distributions.
- Gated Attention Preserves VLA Priors: By ignoring tactile tokens when there is no contact, the VLA's powerful visual-language capabilities remain uncorrupted during free-space movement.
Key Results
- Real-World Success: Across all four bimanual tasks, TacCoRL achieved an average real-world success rate of 72.5%, significantly outperforming the vision-and-imitation-only baseline average of 50.0%.
- Task-Specific Gains: In the Assembly #1 task, the success rate jumped from 35% (baseline) to 95% (TacCoRL), demonstrating a massive improvement in handling occluded, tight-tolerance insertions.
- Ablation Proof: Without the real-data anchor ($\beta=0$), the policy overfit to the simulator, dropping real-world success. A balanced anchor ($\beta=1.0$) yielded the best results, preventing simulator drift while allowing the policy to learn reward-driven contact corrections.
Limitations and Future Work
- Dependency on Real Tactile Data: While it avoids large-scale pretraining, TacCoRL still requires a small set of real tactile demonstrations to serve as the anchor during post-training.
- Manual Simulation Setup: Creating the digital twin (aligning cameras, assets, and identifying controller parameters) currently requires significant manual engineering effort.
- Rigid Contacts Only: The current simulation framework focuses on rigid or near-rigid objects. Tasks involving highly deformable objects or fluids are not yet supported and represent a key direction for future work.
Summary updated