Physics-Informed Tactile-based Shape Estimation of Deformable Linear Objects

Rotem Atari, Stav Ashur, Avishai SintovarXiv 2026 · arXiv

Estimating the shape of Deformable Linear Objects (DLOs)—think cables, wires, and surgical threads—is a notorious "boss level" in robotics. Because these objects can twist and bend in infinite ways, robots usually rely on high-end cameras to track them. But what happens when the robot’s own hand blocks the view, or the cable is being routed through a dark, cramped car chassis?

In this paper, researchers from Tel-Aviv University propose a "vision-free" solution. They demonstrate that by combining high-resolution tactile sensing (touch) with robot proprioception (knowing where its arms are), a robot can reconstruct the full 3D geometry of a cable with surprising accuracy.

TL;DR

The authors introduce a Physics-Informed Vision Transformer (ViT) framework that reconstructs the 3D shape of a cable using only tactile images from the grippers and robot pose data. By embedding physical constraints (like "cables don't stretch") directly into the neural network's loss function, they achieve high-fidelity shape estimation even in environments where cameras are useless.

What Problem Does This Paper Address?

Most DLO manipulation relies on "vision-centric" methods. While effective, vision has a persistent bottleneck: occlusion. In practical tasks like assembly or routing, the robot's grippers often hide the very contact points needed to understand the cable's state.

Alternative methods using Force/Torque (F/T) sensors exist, but they are often "blind" to the local geometry—they can't tell exactly how the cable is oriented inside the grasp. This paper investigates the hypothesis that dense tactile imprints (high-res images of the skin deforming) provide the critical boundary conditions needed to resolve the cable's global 3D configuration without any external cameras.

Methods

The researchers developed a multi-stage architecture called ViT-Sparsh, which fuses touch and pose data:

  1. Tactile Encoder: It uses the Sparsh foundation model, a Vision Transformer (ViT) pretrained on nearly half a million tactile images. This allows the model to extract sophisticated features from the raw "touch images" provided by DIGIT sensors.
  2. Proprioception Encoder: A simple MLP processes the 7D pose of the robot's moving gripper.
  3. Cross-Attention Decoder: This is the "brain" of the system. It uses attention mechanisms to correlate the local tactile gradients at the two contact sites with the global pose to predict the cable's path.
  4. Differentiable B-Spline Layer: Instead of predicting raw points, the model predicts control points for a B-spline. This ensures the output is always a mathematically smooth, continuous curve.
Fig. 2: Architecture of the proposed ViT-Sparsh (ViT-S) -based shape estimation model. The system processes synchronous inputs from two grippers, each providing tactile images and gripper pose data. A ViT-based encoder extracts high-dimensional tactile features which are concatenated with proprioceptive embeddings to form input tokens. These tokens are processed by a cross-attention decoder to capture long-range spatial correlations between the two contact sites. Finally, a differentiable B-spline layer enforces geometric smoothness. The output is a spatial vertex chain representing the DLO configuration.

Crucially, the training is physics-informed. The loss function isn't just looking at distance; it incorporates Discrete Elastic Rods (DER) theory. It penalizes the model if the predicted cable stretches (inextensibility) or bends in physically impossible ways.

Datasets

The authors collected a substantial dataset using a dual-gripper setup and an OptiTrack motion capture system for ground truth:

  • Primary Dataset: 196,678 samples of a 1.5mm braided steel wire.
  • Transfer Datasets: To test generalization, they collected smaller datasets for a 4.5mm USB cable and a longer 2mm steel wire.
  • Data Collection: They varied the grasp orientation (between -45° and 45°) to ensure the model learned to handle "grasp-site ambiguity."

Key Insights and Ideas

  • Touch as a Boundary Condition: The "aha moment" here is that high-res tactile sensors don't just feel pressure; they capture the contact gradient. This gradient acts as a mathematical boundary condition that, when combined with the gripper's position, leaves very few physically valid shapes for the rest of the cable.
  • Foundation Models for Touch: The paper proves that tactile foundation models (like Sparsh), originally designed for discrete tasks like "is this a bolt or a nut?", can be effectively repurposed for complex geometric regression.
  • Physics as a Guardrail: Without the physics-informed loss and the B-spline layer, the model tends to produce "jittery" or physically impossible shapes. Adding these constraints bridges the gap between raw data and physical reality.

Key Results

  • Superior Accuracy: The proposed model achieved a Root Mean Square Error (RMSE) of 30.52 mm, significantly outperforming a "Pose-Only" baseline (55.00 mm). This proves that touch provides vital information that robot position alone cannot.
  • Real-Time Performance: The system runs at 16 Hz, which is fast enough for real-time robotic control loops.
  • Robustness: The model maintained stable performance even when the tactile images were artificially corrupted with noise.
  • Transferability: The model demonstrated "sample-efficient transfer." By fine-tuning a model trained on steel wire with just a few thousand samples of a USB cable, it achieved better results than training a new model from scratch.
Visual examples showing the predicted shape (magenta) vs. ground truth (cyan) for different DLOs.

Limitations and Future Work

While impressive, the system has a few constraints:

  • Quasi-Static Assumption: The model assumes the cable is moving slowly. It doesn't currently account for high-speed "whipping" or inertial effects.
  • Fixed Length: The current architecture assumes the total length of the DLO is known and constant.
  • Extreme States: The authors noted performance degradation during "out-of-distribution" events, such as the cable nearly buckling or slipping out of the gripper.

Future work aims to move toward Sim2Real (training in simulation to reduce the need for manual data collection) and incorporating dynamic length sensing to make the robot even more adaptable to different cables on the fly.

Summary updated

Abstract

Accurate shape estimation of Deformable Linear Objects (DLOs) is a fundamental challenge in robotic manipulation, often hindered by visual occlusions and the high-dimensional configuration space of flexible materials. While Force/Torque (F/T) and vision-centric methods have been proposed, they frequently struggle with grasp-site ambiguities and environmental interference. In this letter, we investigate the hypothesis that global shape estimation can be achieved through tactile sensing and proprioception, without any visual perception. We contend that dense, high-resolution tactile imprints capture critical boundary conditions and contact gradients that are sufficient to resolve the high-dimensional configuration of the DLO. To prove this, we introduce a Physics-Informed Vision Transformer (ViT) framework that maps tactile images to a 3D vertex chain. By leveraging self-attention mechanisms and a physics-weighted loss function, our model enforces structural priors to ensure that the predicted geometry adheres to the object's physical manifold. Experimental results demonstrate that our tactile-based approach achieves superior reconstruction accuracy across Euclidean and Chamfer metrics. Furthermore, we demonstrate the efficiency of this modality, achieving model transfer to novel DLO materials and geometries. Our findings suggest that tactile feedback provides a robust and computationally efficient alternative for DLO state estimation in visually degraded or occluded environments.
BibTeX
@misc{atari2026physics,
  author = {Rotem Atari and Stav Ashur and Avishai Sintov},
  title = {{Physics-Informed Tactile-based Shape Estimation of Deformable Linear Objects}},
  year = {2026},
  eprint = {2611.22990},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.22990},
}
Figure