Exploring Proximity and Tactile Sensing for Robotic Manipulation

Eric T. Chang, Bianca Aumann, Zhanpeng He, Mohamed Elnoor, Thomas Liang, Matei Ciocarlie, Nima Fazeli, Joshua Migdal, Tye Brady, Taskin PadirarXiv 2026 · arXiv

TL;DR

ToFTouch is a novel multimodal sensor that bridges the gap between vision and touch by integrating multizone Time-of-Flight (ToF) proximity sensing with high-frequency tactile feedback. By providing a "fingertip point-of-view" that anticipates contact before it happens, the sensor enables robots to reduce grasp cycle times by 7.4% and successfully manipulate objects in cluttered environments where traditional cameras are occluded.

What Problem Does This Paper Address?

In robotic manipulation, there is a fundamental "sensing gap" between seeing an object and touching it. Traditional tactile sensors are reactive—they only provide information after contact has occurred. Conversely, while vision systems (like wrist-mounted cameras) provide global context, they often suffer from occlusion, high latency, and lighting sensitivity as the gripper approaches an object.

This paper introduces ToFTouch to provide "pre-touch" sensing. By mounting multizone proximity sensors directly on the fingertips, the robot can perceive local geometry and distance in real-time during the approach phase. This allows for anticipatory actions—like pre-shaping a gripper or adjusting for depth uncertainty—that are difficult to achieve with vision or tactile sensing alone.

Methods

The researchers developed a compact (45 × 30 × 10 mm) sensing module that combines two distinct modalities:

  1. Multizone Proximity Sensing: The module utilizes three VL53L8CX ToF sensors. Unlike standard single-point ToF sensors, these provide a 4×4 depth array (16 zones) at 60 Hz. By orienting these sensors at different angles (e.g., 0°, 45°, and 90°), the system achieves broad spatial coverage around the fingertip.
  2. Optics-Based Tactile Sensing: A 3×3 cm patch uses four LEDs and four photodiodes to measure the deformation of a reflective elastomer. This tactile array operates at 300 Hz, providing high-frequency feedback once contact is made.

The hardware is built on a flexible PCB to accommodate the folded sensor arrangement and is integrated with a Teensy 4.1 microcontroller using micro-ROS for communication.

Fig. 1: Exploded view of ToFTouch. The lower inset compares the sensor size to a U.S. penny. The upper inset shows an example of ToF data accumulated over time during a grasp trajectory with the sensor mounted on a parallel gripper. Red points are from the sensor pointing down and blue points are from the sensor pointing at a 45° angle towards the object.

Datasets

The paper does not rely on standard large-scale machine learning datasets. Instead, the authors evaluated the sensor through three targeted robotic manipulation experiments using a Franka FR3 arm and a Robotiq parallel jaw gripper:

  • Pre-grasp adjustment: Testing cycle time reduction during approach to a cuboidal box.
  • Item extraction: Retrieving a box from clutter with simulated depth offsets of ±1 cm and ±2 cm.
  • Dense insertion: Inserting an oblong box into a tight cavity between other objects to monitor extrinsic contacts and available space.

Key Insights and Ideas

  • Anticipatory Gripper Closure: By using proximity data to detect the table surface at a 130 mm threshold, the robot can begin closing its gripper during the approach motion rather than waiting to reach a static coordinate. This mimics human-like reaching behavior.
  • Overcoming Occlusion: Fingertip-mounted sensors move with the hand, maintaining a local view of the contact area even when the robot's arm or the object itself occludes external cameras.
  • Local Geometry Mapping: The 4×4 multizone grid allows the robot to distinguish between surfaces and edges. By accumulating these points over time, the system can generate localized point clouds of the environment during a trajectory.
  • Continuous Sensing Stream: The fusion of ToF and tactile sensing creates a continuous data stream from approach through contact, allowing for smoother transitions in control loops.
Visualization of accumulated ToF data points forming a localized point cloud of a target object.

Key Results

  • Efficiency: ToF-enabled anticipatory closure reduced the average grasp cycle time by 7.4% (a saving of ~0.26 seconds per grasp).
  • Robustness to Uncertainty: In extraction tasks with a ±2 cm depth error, the baseline (no ToF) failed completely. ToFTouch achieved a 90% success rate, correcting the robot's depth in real-time with a standard deviation of less than 0.4 cm.
  • Insertion Monitoring: During dense packing, the sensor successfully mapped the "empty cavity" and detected extrinsic contacts via the tactile patch, providing data that vision-only systems would lose due to proximity.

Limitations and Future Work

The authors identified several areas for improvement:

  • Thermal Management: The primary failure mode was sensor overheating, which led to corrupt ToF readings. Future designs will need better heat dissipation.
  • Calibration: There was a noted discrepancy between the "Mean Δ" (measured deviation) and the actual physical offset, likely due to slight misalignments during manual mounting and a lack of a formal calibration step.
  • Close-Range Blind Spot: The ToF sensors become unreliable at distances less than 1–2 cm, creating a small "blind zone" just before tactile contact occurs.
  • Future Directions: The team aims to explore learned control policies that can more effectively leverage this rich spatial data and deploy the sensors on multi-fingered humanoid hands.

Summary updated

Abstract

Pre-touch sensing, or sensing information about an object before contact, is an attractive modality for robot hands as it can allow a robot to anticipate and respond to a contact before it happens. In this work, we introduce ToFTouch, a multimodal tactile sensor combining three multizone (4$\times$4 resolution) time-of-flight sensors with an optics-based pressure sensing array. We mount this sensor on a parallel jaw gripper and demonstrate its utility on three distinct tasks: (i) pre-grasp adjustment, (ii) item extraction from clutter, and (iii) item insertion into tight spaces. For pre-grasp adjustment, ToF feedback reduced cycle time by 7.4$\%$ by enabling anticipatory gripper closure during approach. For extraction, ToF sensing enabled successful retrieval across all tested depth offsets ($\pm$2 cm) deviating from the nominal pick point, including conditions where baseline approaches failed, with standard deviations below 0.4 cm. For insertion into dense clutter, we explore the proximity and tactile information provided from ToFTouch during an insertion trajectory. The results demonstrate the utility of multizone ToF sensing in a manipulation environment.
BibTeX
@misc{chang2026exploring,
  author = {Eric T. Chang and Bianca Aumann and Zhanpeng He and Mohamed Elnoor and Thomas Liang and Matei Ciocarlie and Nima Fazeli and Joshua Migdal and Tye Brady and Taskin Padir},
  title = {{Exploring Proximity and Tactile Sensing for Robotic Manipulation}},
  year = {2026},
  eprint = {2611.03489},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.03489},
}
Figure