BIFTA: Brain-Inspired Few-Shot Tactile Adaptation for Unknown Sensors

Boheng Liu, Ziyu Li, Xia WuarXiv 2026 · arXiv

TL;DR

Tactile foundation models often suffer a catastrophic performance collapse when deployed on new, unseen sensor hardware due to variations in optics and elastomer mechanics. To fix this without costly retraining, BIFTA (Brain-Inspired Few-Shot Tactile Adaptation) introduces a training-free framework that rapidly recalibrates frozen encoder features using a tiny labeled dataset. By modifying the feature space geometry rather than the model weights, BIFTA boosts cross-sensor accuracy from near-chance levels (6.86%) up to 87.09% in a fraction of a second.

What Problem Does This Paper Address?

For robots to interact deftly with the physical world, touch is just as critical as vision. Vision-based tactile sensors (like DIGIT or GelSight) capture high-resolution contact geometry by imaging how an internal elastomer deforms. However, hardware heterogeneity is a massive hurdle. Unlike standard cameras, different tactile sensors vary wildly in their internal lighting, marker layouts, fields of view, and mechanical properties.

As a result, a tactile model trained on one sensor type completely breaks down when plugged into a different, unknown sensor type.

Figure 1: TVL performs well on the source sensor but collapses on unknown sensors. A linear classifier trained on the source sensor is evaluated with frozen TVL features and no target labels. Green bars denote seen source sensors and red bars denote unseen target sensors. Bars show mean accuracy over three seeds. On SITR, accuracy falls from 71.75% on DIGIT to 7.58% on average across six unknown sensors. On TacVerse Shape, it falls from 38.78% on GelSightNoMarker to 9.56% across five unknown sensors.

Traditional solutions like fine-tuning the entire network require significant target data and computation, and they risk overfitting or erasing previously learned general representations. The authors ask: How can a tactile model rapidly and reliably transfer its perception capabilities to an entirely new sensor using only a handful of labeled examples?

Methods

Inspired by how the human brain combines long-term memory with rapid sensory adaptation, BIFTA avoids gradient descent entirely. Instead, it treats cross-sensor transfer as a support-conditioned inference problem over a frozen tactile encoder.

Figure 2: Information flow through BIFTA. A frozen encoder produces two readouts. Rapid support memory forms the probability anchor P0, while the support-conditioned spectral graph reshapes query geometry and constructs the cross-readout affinity matrix G. Prediction ambiguity and readout disagreement determine the query-wise reliability gate R. Anchored recurrent inference combines P0, G, and R to produce class probabilities or identity rankings.

The framework consists of three tightly coordinated modules:

  1. Rapid Support Memory: BIFTA extracts two distinct feature readouts from different layers of the frozen encoder. It uses the few-shot labeled "support" data to fit a shrinkage Linear Discriminant Analysis (LDA) classifier for each readout, fusing them into a stable initial probability anchor.
  2. Support-Conditioned Spectral Geometry: Sensor shifts distort feature neighborhoods, amplifying non-discriminative variations. BIFTA calculates the within-class covariance of the support set and applies a spectral transform to suppress these unstable, sensor-dependent feature directions. It then constructs a cross-readout consensus graph to map relations between unlabeled query samples.
  3. Reliability-Gated Anchored Recurrence: To safely propagate information across the graph, BIFTA measures uncertainty using two metrics: prediction ambiguity (entropy) and readout disagreement. Queries with high uncertainty rely heavily on their neighbors' graph evidence, while highly certain queries stick close to their support anchors. This evidence is iteratively diffused using an optimal-transport-style (Sinkhorn) normalization.

Datasets

BIFTA was rigorously evaluated across three distinct tactile datasets using two state-of-the-art frozen backbones (TVL and Sparsh):

  • SITR: A 16-class object dataset. The source sensor is DIGIT, and the model adapts to six unknown target sensors (four GelSight Mini variants, GelSight Hex, and GelSight Wedge).
  • TacVerse Shape: A 9-class shape geometry dataset. The source is GelSightNoMarker, adapting to five target sensors (MagicGripper, MagicTac, TacTip, ViTac, and ViTacTip).
  • TacQuad: A closed-set identity ranking and retrieval dataset containing 56 enrolled object identities across three RGB sensors (DIGIT, GelSight Mini, and DuraGel).

Key Insights and Ideas

  • Geometry Over Optimization: Instead of updating model weights—which is slow and data-hungry—BIFTA updates the geometry of the feature space. It mathematically "cleans" the features to strip away sensor-specific artifacts.
  • Disagreement as an Uncertainty Signal: By comparing two separate readouts from the same frozen backbone, BIFTA can inherently tell when a sensor shift has confused the model. If the readouts disagree, the model knows to trust graph-based neighbor evidence over its own initial guess.
  • Spectral Normalization is King: The ablation studies revealed that removing spectral normalization caused the largest performance drop. Suppressing high-variance, sensor-dependent feature dimensions is absolutely vital to restoring clean neighborhood structures.

Key Results

BIFTA achieved massive performance leaps over standard few-shot methods (like SimpleShot and LaplacianShot) and recent cross-sensor calibration techniques:

Table 1: Few-shot classification on six unknown SITR sensors with two pretrained backbones under different label budgets. Accuracy (%) is averaged across the unknown sensors and reported as mean ± sample standard deviation over three random seeds. I/T denotes inductive/transductive inference. Best results are in bold and second-best results are underlined.
  • Massive Accuracy Recovery: On the SITR benchmark with a 10% label budget, BIFTA raised the mean accuracy of a frozen Sparsh backbone from 6.86% to 87.09%, outperforming the next best baseline by 47.22 percentage points.
  • Robust Generalization: On TacVerse Shape, the same budget raised Sparsh accuracy from 13.67% to 83.58%.
  • High Efficiency: Because it requires no backpropagation, BIFTA adapts to a new sensor in less than a second (e.g., ~0.28 seconds for Sparsh on SITR), making it highly practical for real-world robotic deployment.
Figure 3: Performance comparison with the strongest baseline across sensors on SITR and TacVerse. Each cell reports the accuracy gain of BIFTA for the corresponding unknown sensor and label budget.

Limitations and Future Work

While highly effective, BIFTA currently processes unlabeled query data in offline batches. The authors note that real-world robotics will require adapting the framework for online, streaming query data where decisions must be made instantly on a single incoming frame.

Additionally, future work will focus on optimizing graph construction using sparse neighborhoods to maintain low computational overhead during continuous, long-term robot manipulation tasks.

Summary updated

Abstract

Advances in tactile sensing have made contact-rich perception possible, accelerating progress in robotic manipulation, material understanding, and embodied interaction. However, because optical design, elastomer mechanics, and imaging geometry differ substantially across tactile sensors, models trained on known sensor types can suffer an abrupt performance collapse on unknown sensors. To address this problem, we propose the Brain-Inspired Few-Shot Tactile Adaptation (BIFTA) framework; it draws on the brain's rapid sensory adaptation mechanism to adapt a frozen encoder to an unknown tactile sensor from a small labeled support set. BIFTA preserves pretrained representations through dual-view statistical memory, constructs support-conditioned spectral graphs to repair sensor-dependent feature neighborhoods, and applies uncertainty-gated recurrent propagation to strengthen reliable cross-query evidence. Extensive benchmarks across three tactile datasets show that BIFTA substantially improves adaptation to unknown sensors: with only 10\% labeled target data on SITR, it raises mean Sparsh accuracy from 6.86\% for the frozen source classifier to 87.09\%, exceeding the strongest implemented prior comparison by 47.22 percentage points, and these gains generalize across datasets, pretrained backbones, and tactile tasks. These results validate BIFTA for data-efficient adaptation to unknown tactile sensors and offer a promising route toward tactile models that transfer across heterogeneous hardware.
BibTeX
@misc{liu2026bifta,
  author = {Boheng Liu and Ziyu Li and Xia Wu},
  title = {{BIFTA: Brain-Inspired Few-Shot Tactile Adaptation for Unknown Sensors}},
  year = {2026},
  eprint = {2609.08673},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2609.08673},
}
Figure