HumanFT: A Human-like Fingertip Multimodal Visuo-Tactile Sensor

Yifan Wu, Yuzhou Chen, Zijian Zhu, Xuhao Qin, Chenxi Xiao

Abstract

Tactile sensors play a crucial role in enabling robots to interact effectively and safely with objects in everyday tasks. In particular, visuotactile sensors have seen increasing usage in two and three-fingered grippers due to their high-quality feedback. However, a significant gap remains in the development of sensors suitable for humanoid robots, especially five-fingered dexterous hands. One reason is because of the challenges in designing and manufacturing sensors that are compact in size. In this paper, we propose HumanFT, a multimodal visuotactile sensor that replicates the shape and functionality of a human fingertip. To bridge the gap between human and robotic tactile sensing, our sensor features real-time force measurements, high-frequency vibration detection, and overtemperature alerts. To achieve this, we developed a suite of fabrication techniques for a new type of elastomer optimized for force propagation and temperature sensing. Besides, our sensor integrates circuits capable of sensing pressure and vibration. These capabilities have been validated through experiments. The proposed design is simple and cost-effective to fabricate. We believe HumanFT can enhance humanoid robots' perception by capturing and interpreting multimodal tactile information.

Introduction

While visuotactile sensors like GelSight have revolutionized robotic manipulation, they face significant hurdles for humanoid applications. Most are too bulky to fit into the slim, anthropomorphic fingers of a dexterous hand (typically 18-20 mm in diameter) and often focus solely on 3D shape or contact deformation. Human hands, by contrast, are multimodal: we perceive pressure, detect fine textures through high-frequency vibration, and react to temperature.

HumanFT addresses these challenges by packing diverse sensing capabilities into a form factor that mimics the human distal phalanx (12x20x35 mm). For a manufacturing cost of approximately $50, it provides a "super-sense" of 3D shape, multi-axis force, high-frequency vibrations, and overtemperature alerts.

Sensor Design and Methodology

The HumanFT sensor is a hybrid of optical and electronic sensing components integrated into a compact housing.

Exploded view of the HumanFT sensor and detailed PCB layout showing integrated multimodal sensors.

Core Components

  • Solid Elastomer Body: Unlike traditional hollow designs, HumanFT uses a solid, deformable PDMS body. This allows contact forces to propagate directly through the material to the base. The surface is coated with a Lambertian reflective layer mixed with thermochromic pigments that change color at high temperatures.
  • Endoscopic Camera: A tiny camera with a 140° field of view sits inside the finger, monitoring elastomer deformation. It uses photometric stereo techniques to reconstruct the 3D shape of touched objects.
  • Integrated PCB: The base of the elastomer houses a custom PCB containing four pressure sensors for multi-axis force measurement ($F_x, F_y, F_z$), a MEMS microphone for high-frequency vibration detection (up to 1 kHz), and LEDs for internal illumination.

Training and Reconstruction

To enable 3D reconstruction, a "digital twin" approach was employed. A PyBullet simulation rendered ground-truth normal maps of various indentations, which were combined with real-world data captured using 3D-printed indenter molds. A Multilayer Perceptron (MLP) was then trained to map RGB pixel intensities from the camera to these surface normals.

Key Insights and Results

By utilizing a solid block of elastomer rather than a thin membrane, the sensor enables direct force propagation to the underlying pressure sensors. This allows the system to "feel" pressure even when the camera is focused on surface details. Furthermore, the integration of thermochromic paint provides a low-cost temperature safety signal; the camera detects a color shift to yellow at 65°C within approximately one second.

The physical HumanFT sensor prototype, demonstrating its compact, human-fingertip scale.

Experimental validation confirmed the sensor's high performance across multiple modalities:

  • Compactness: At $12 \times 20 \times 35$ mm, it is one of the smallest multimodal visuotactile sensors developed to date.
  • Force and Vibration: The sensor demonstrated a linear relationship between applied force and pressure readings. In sliding tests over ridges, the MEMS microphone successfully captured distinct frequency peaks, allowing the robot to differentiate between surface roughness levels that exceed the camera's frame rate capabilities.

Conclusion and Future Work

HumanFT represents a significant step toward human-like tactile perception for humanoid robots. While the current prototype is highly capable, future work will address challenges such as thermal drift caused by LED heat and the complexity of 3D reconstruction on curved, solid surfaces. The researchers plan to integrate these sensors into a full five-fingered dexterous hand to evaluate how multimodal data improves complex manipulation tasks in real-world environments.

Summary updated

Abstract

Tactile sensors play a crucial role in enabling robots to interact effectively and safely with objects in everyday tasks. In particular, visuotactile sensors have seen increasing usage in two and three-fingered grippers due to their high-quality feedback. However, a significant gap remains in the development of sensors suitable for humanoid robots, especially five-fingered dexterous hands. One reason is because of the challenges in designing and manufacturing sensors that are compact in size. In this paper, we propose HumanFT, a multimodal visuotactile sensor that replicates the shape and functionality of a human fingertip. To bridge the gap between human and robotic tactile sensing, our sensor features real-time force measurements, high-frequency vibration detection, and overtemperature alerts. To achieve this, we developed a suite of fabrication techniques for a new type of elastomer optimized for force propagation and temperature sensing. Besides, our sensor integrates circuits capable of sensing pressure and vibration. These capabilities have been validated through experiments. The proposed design is simple and cost-effective to fabricate. We believe HumanFT can enhance humanoid robots' perception by capturing and interpreting multimodal tactile information.
BibTeX
@inproceedings{wu2025humanft,
  author = {Yifan Wu and Yuzhou Chen and Zijian Zhu and Xuhao Qin and Chenxi Xiao},
  title = {{HumanFT: A Human-like Fingertip Multimodal Visuo-Tactile Sensor}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2025},
  doi = {10.1109/icra55743.2025.11128300},
}
Figure