TapTacTouch: Shape Completion from Sparse Tactile Contacts with Geometric Priors

Yoobin Shin, Jaeyoung Lee, Jieun Kang, Hyun Song, Woojin Kim, Andrew Jaeyong ChoiarXiv 2026 · arXiv

TL;DR

TapTacTouch is a vision-free, training-free framework that reconstructs 3D object meshes using only sparse tactile data from a robotic hand. By integrating multi-scale Gaussian Processes with observation-driven geometric priors, it can accurately complete the shape of previously unseen objects without needing cameras or pre-trained models.

What Problem Does This Paper Address?

Most robots "see" the world through cameras. However, visual perception often fails in real-world manipulation: a robotic hand might occlude the very object it is trying to grasp, or the environment might be too dark or cluttered for a clear view.

Tactile sensing is a natural alternative, but it presents a "sparsity" challenge. While a camera provides millions of pixels at once, a robot only gets a few points of contact each time it touches an object. Reconstructing a full, continuous 3D shape from these few "pokes" is mathematically difficult. Existing solutions often rely on massive datasets of 3D shapes to "guess" the missing parts, but these models fail when they encounter an object category they haven't seen before.

Methods

The authors propose TapTacTouch, a pipeline that moves from physical contact to a completed 3D mesh using three main components:

  1. Tactile Data Acquisition: A 6-DoF robotic arm with a 20-DoF multi-fingered hand performs a series of "scans" (top and sides). It uses 3×3 pressure sensor arrays on the finger pads to record contact points.
  2. Multi-scale GPIS: They use Gaussian Process Implicit Surfaces (GPIS) to turn discrete points into a continuous surface. Crucially, they employ a multi-scale kernel that uses a short length-scale to capture fine local details and a long length-scale to maintain global smoothness.
  3. Geometric Priors (PCG & CSP): To fill the gaps where the robot didn't touch, the system generates Pseudo-Contacts (PCG) by interpolating between nearby real contacts. It also constructs a Cross-Section Prior (CSP)—a polygonal prism based on the observed side contacts—to provide structural guidance in unobserved regions.
Overview of the TapTacTouch pipeline, showing tactile exploration, the multi-scale GPIS reconstruction process, and the final mesh.

Datasets

Since the method is training-free, no large-scale training datasets were used. For evaluation, the researchers used a set of six 3D-printed objects with varying geometric complexities (e.g., cylinders, wedges, and asymmetric shapes). These objects were mounted at a fixed height to allow for consistent tactile exploration.

Key Insights and Ideas

  • Training-Free is Category-Agnostic: Because the system doesn't rely on a database of "known" shapes, it can reconstruct any solid object, regardless of whether it's a common tool or a completely novel geometry.
  • Priors from Observations, Not Data: Instead of using a neural network to predict missing parts, the Cross-Section Prior (CSP) uses the robot's own measurements to create a "best guess" geometry. This ensures the reconstruction stays grounded in physical reality rather than hallucinating features.
  • Multi-Scale Balance: The use of two different kernels allows the system to be "stiff" enough to form a solid object while remaining "flexible" enough to capture small surface irregularities.

Key Results

The framework was validated through real-world experiments, producing impressive quantitative results:

  • Accuracy: Achieved an average Chamfer Distance (CD) of 1.610 mm and an F-Score of 95.00, indicating high fidelity to the ground-truth shapes.
  • Efficiency: Once the tactile data is collected, the 3D mesh is generated in an average of 49.99 seconds.
  • Robustness: The system successfully reconstructed all six test objects, even in regions where contact was extremely sparse, thanks to the geometric priors.
Fig. 1. Overview of the proposed tactile-only 3D reconstruction framework. Tactile contact points are aggregated into a 3D point cloud and reconstructed into a 3D mesh using Gaussian Process Implicit Surfaces (GPIS).

Limitations and Future Work

While the reconstruction itself is fast (~50s), the physical exploration—the time the robot takes to move and touch the object—takes about 30 minutes per object. The current exploration policy is "blind" and follows a fixed sequence.

The authors suggest that future work should focus on adaptive touch: a policy where the robot identifies "uncertain" regions in the reconstruction and proactively moves to touch those specific areas, significantly reducing the number of contacts needed and speeding up the overall process. Additionally, the current system assumes a single, stationary solid object; handling moving or deformable objects remains a challenge.

Summary updated

Abstract

3D object reconstruction is essential for robotic perception and manipulation, yet conventional approaches predominantly rely on visual observations that can be limited by occlusion or unavailable in visually constrained environments. In this work, we propose a vision-free, training-free, and category-agnostic framework for reconstructing 3D object meshes from sparse tactile contacts without relying on pretrained shape models. A 6-DoF robotic arm equipped with a 20-DoF multi-finger robotic hand collects tactile observations using 3x3 pressure sensor arrays mounted on the finger pads. The acquired tactile measurements are transformed into 3D contact constraints through robot kinematics and integrated into a multi-scale Gaussian Process Implicit Surface (GPIS). To address sparse and incomplete surface observations, the proposed method incorporates contact confidence weighting, weak pseudo-surface constraints, and an observation-driven geometric prior to preserve local surface geometry and provide structural guidance in unobserved regions. Real-world experiments on six objects achieve a Chamfer Distance (CD) of 1.610, an Earth Mover's Distance (EMD) of 7.738, and an F-Score of 95.00, with reconstruction completed in an average of 49.99 s per object. These results demonstrate that sparse tactile sensing can provide sufficient geometric information for continuous 3D object reconstruction without visual or semantic information.
BibTeX
@misc{shin2026taptactouch,
  author = {Yoobin Shin and Jaeyoung Lee and Jieun Kang and Hyun Song and Woojin Kim and Andrew Jaeyong Choi},
  title = {{TapTacTouch: Shape Completion from Sparse Tactile Contacts with Geometric Priors}},
  year = {2026},
  eprint = {2611.24207},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.24207},
}
Figure