TacBridge: Learning Unified Tactile Representations through Cross-Sensor Correspondence

Yongji Fu, Yijiong Lin, Yi Zhou, Yi Qi, Guanqun CaoarXiv 2026 · arXiv

TL;DR

TacBridge is a new framework that creates a "universal language" for diverse optical tactile sensors by training on paired videos of the same contact events. By learning to translate interactions between different sensor types (like GelSight and DIGIT), it achieves state-of-the-art performance in robotic perception tasks and enables high-fidelity cross-sensor video generation.

What Problem Does This Paper Address?

In the world of robotic touch, we have a "Tower of Babel" problem. Different optical tactile sensors—such as GelSight, DIGIT, and GelSlim—all record contact differently. They vary in illumination, elastomer material, and camera geometry. Consequently, a robot trained to recognize a silk fabric using a DIGIT sensor might be completely lost if you swap it for a GelSight Mini.

This fragmentation makes it incredibly difficult to combine datasets or transfer learned skills across different hardware. While previous research attempted to align these sensors by looking at broad data distributions, TacBridge argues that we need a more direct approach: event-level correspondence. If we show a model exactly how the same physical interaction looks through two different sensors, it can learn to separate the "content" of the touch from the "style" of the device.

Methods

The authors introduce a three-stage pipeline to bridge the gap between sensors:

  1. Multi-Sensor Pre-training: They first train a Spatiotemporal Masked Autoencoder (MAE) on a massive corpus of tactile videos from various sensors. The model learns to "fill in the blanks" of masked video patches, forcing it to understand both spatial contact patterns and how they evolve over time.
  2. Event-Paired Learning: This is the core innovation. Using datasets where the same contact event is recorded by multiple sensors (or simulated across multiple sensor profiles), the model is trained to reconstruct a "target" sensor's view using only the "source" sensor's data. This teaches the encoder to extract sensor-invariant physical features.
  3. Controllable Tactile Generation: To prove the representation is truly unified, they build a TacDiT (Tactile Diffusion Transformer). This latent flow model can take a video from one sensor and "render" it as if it were seen by another, conditioned on the target sensor's identity and background.
The TacBridge framework: (1) Pre-training on masked videos, (2) Controllable generation via a latent flow model, and (3) Downstream applications in rendering and perception.

Datasets

The study leverages a mix of real-world and simulated data:

  • TacQuad: A real-world dataset providing synchronized observations from DIGIT, GelSight Mini, and DuraGel sensors.
  • ToucHD-Sim: A simulation environment that renders the same interaction under nine different optical sensor appearances.
  • Pre-training Corpus: A massive collection of public datasets including Touch and Go, VisGel, and ObjectFolder Real, covering a wide array of objects and sensors.
  • Real-Robot Deployment: A custom dataset of a robot arm pressing 50 different fabrics to test real-world generalization.

Key Insights and Ideas

  • Correspondence is King: The authors show that explicitly linking paired observations is far more effective than just training on a mix of unpaired data. It provides a "ground truth" for what the underlying physics should look like regardless of the camera.
  • Temporal Dynamics Matter: By using tactile videos rather than static images, TacBridge captures the "evolution" of a touch (establishment, maintenance, and release). This proved critical for dynamic tasks like slip detection.
  • Latent Translation: The ability to translate features in a shared latent space allows for "zero-shot" style transfer between sensors. You can effectively "see" through a sensor you don't even own by translating your current data into its domain.

Key Results

TacBridge outperformed strong baselines (like AnyTouch and Sparsh) on 9 out of 11 downstream metrics.

  • Dynamic Perception: The most dramatic gains were in dynamic tasks. For example, on the DIGIT sensor, slip detection F1-scores jumped from 40.60 (AnyTouch) to 85.77 (TacBridge).
  • Force Estimation: RMSE for force prediction was reduced by 17% to 69% compared to the strongest baselines.
  • Real-World Generalization: When tested on a real robot for fabric attribute classification (e.g., is it "fuzzy" or "stiff"?), the model achieved 73% accuracy on entirely unseen fabrics, demonstrating that the learned features generalize well beyond the training set.

Limitations and Future Work

While TacBridge is a significant step toward a "Foundation Model" for touch, the authors note several areas for growth:

  • Sensor Variety: The current evaluation covers a limited range of sensors. Expanding to more diverse hardware (like capacitive skins) is a logical next step.
  • Real-World Fidelity: While the cross-sensor translation works well in simulation, achieving the same level of photorealism on real-to-real sensor pairs remains a challenge.
  • Closed-Loop Control: The current work focuses on "offline" perception. The ultimate goal is to integrate these unified representations into real-time, closed-loop robotic manipulation.

For more details, you can find the full paper "TacBridge: Learning Unified Tactile Representations through Cross-Sensor Correspondence" on arXiv.

Summary updated

Abstract

Optical tactile sensors record tactile videos that capture local deformation and its temporal evolution during contact, supporting robot manipulation, force estimation, and tactile representation learning. Differences in imaging geometry, background appearance, illumination, resolution, and deformation response lead to sensor-dependent observations of the same contact across GelSight, DIGIT, GelSlim, TacTip, and other optical tactile sensors. This variability limits representation transfer and the joint use of multi-sensor data. We introduce TacBridge, a framework for learning unified representations from multi-sensor tactile videos through cross-sensor correspondence. Specifically, multi-sensor masked tactile-video pre-training learns spatiotemporal representations across sensors. We then train the same masked model jointly on same-sensor and cross-sensor reconstruction, using paired source and target videos of matched contact events. Experiments on cross-sensor translation and downstream perception demonstrate the cross-sensor generalization and task transferability of the learned tactile representations.
BibTeX
@misc{fu2026tacbridge,
  author = {Yongji Fu and Yijiong Lin and Yi Zhou and Yi Qi and Guanqun Cao},
  title = {{TacBridge: Learning Unified Tactile Representations through Cross-Sensor Correspondence}},
  year = {2026},
  eprint = {2611.20939},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.20939},
}
Figure