2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

Daniel Fernandes Gomes, Zhonglin Lin, Shan LuoOpenalex Unknown 2020 · DOI · arXiv · OpenAlex

Abstract: Sensing contacts throughout the fingers is an essential capability for a robot to perform manipulation tasks in cluttered environments. However, existing tactile sensors either only have a flat sensing surface or a compliant tip with a limited sensing area. In this paper, we propose a novel optical tactile sensor, the GelTip, that is shaped as a finger and can sense contacts on any location of its surface. The sensor captures high-resolution and color-invariant tactile images that can be exploited to extract detailed information about the end-effector's interactions against manipulated objects. Our extensive experiments show that the GelTip sensor can effectively localise the contacts on different locations of its finger-shaped body, with a small localisation error of approximately 5 mm, on average, and under 1 mm in the best cases. The obtained results show the potential of the GelTip sensor in facilitating dynamic manipulation tasks with its all-round tactile sensing capability.

Introduction

Most high-resolution tactile sensors currently used in robotics are "flat-panel" designs, functioning much like a human fingertip pad. While these are excellent for sensing objects directly within a robot's grasp, they leave the robot "blind" to contacts occurring on the sides or back of the finger. In real-world scenarios—such as a robot reaching into a crowded drawer or navigating a cluttered shelf—collisions and interactions frequently happen on the sides of the digits.

The GelTip sensor mounted on a robotic gripper, showing the high-resolution tactile image captured during contact with a strawberry.

Existing "all-around" sensors often rely on low-resolution electronic skins that struggle to capture fine geometric details. The GelTip addresses this gap by adapting the high-resolution imaging principles of GelSight sensors into a 360-degree, finger-shaped form factor. This allows for precise contact localization from any direction, enabling more sophisticated and safer manipulation in complex environments.

Sensor Design and Fabrication

The GelTip's design is an elegant combination of optics and material science. The core of the sensor is a rigid transparent tube, acting as a structural "bone." This tube is covered by a soft elastomer membrane and coated with a layer of opaque, reflective paint. Inside the tube, a wide-angle (170°) camera and a set of LEDs are mounted at the base.

Exploded view of the GelTip sensor assembly, showing the layered construction from the external membrane to the internal camera housing.

When an object touches the finger, it indents the elastomer. The internal camera captures these deformations as changes in light and shadow on the internal paint layer. Because the sensing surface is curved (a cylinder capped with a hemisphere), the authors derived a specific projective function to map 2D pixels from the fisheye camera back to 3D coordinates on the finger’s surface. Through experimentation, the team found that a specific 1:22:22 silicone-to-tackifier ratio provided the ideal balance of softness and the ability to capture fine details, such as fingerprints.

Key Insights and Performance

Achieving "all-around" sensing required overcoming several optical challenges. A standard 70° camera lens can only see the very tip of the finger; a 170° wide-angle lens was mandatory to capture the sides of the cylindrical body. Additionally, traditional aluminum powder paint created "dark zones" at the tip due to the curvature, leading the researchers to use a metallic elastic paint that provides more uniform illumination.

Localization Accuracy

The sensor was evaluated using a custom benchmark involving seven 3D-printed shapes (cones, spheres, prisms, etc.) tapped at various orientations. Key results include:

  • Average Localization Error: Approximately 5 mm across all test cases.
  • Precision Peaks: In optimal configurations (specifically at a $\pi/4$ angle), the error dropped to just 1.04 mm.
  • Geometry Sensitivity: The sensor is significantly more accurate at localizing "sharp" contacts (like a tube or cone) than "diffuse" ones (like a large sphere). Sharp points create higher-contrast imprints that are easier for the vision system to process.

Conclusion and Future Work

The GelTip demonstrates that high-resolution, all-around tactile sensing is achievable in a compact, finger-like form factor. While current localization relies on basic image processing, the authors suggest that deep learning models (CNNs) could further improve accuracy and help estimate contact force. Future research will focus on using the GelTip for dynamic tasks, such as detecting slip or re-orienting objects within a grasp using the side of the finger, moving beyond static contact detection to active, tactile-driven manipulation.

Summary updated

Abstract

Sensing contacts throughout the fingers is an essential capability for a robot to perform manipulation tasks in cluttered environments. However, existing tactile sensors either only have a flat sensing surface or a compliant tip with a limited sensing area. In this paper, we propose a novel optical tactile sensor, the GelTip, that is shaped as a finger and can sense contacts on any location of its surface. The sensor captures high-resolution and color-invariant tactile images that can be exploited to extract detailed information about the end-effector's interactions against manipulated objects. Our extensive experiments show that the GelTip sensor can effectively localise the contacts on different locations of its finger-shaped body, with a small localisation error of approximately 5 mm, on average, and under 1 mm in the best cases. The obtained results show the potential of the GelTip sensor in facilitating dynamic manipulation tasks with its all-round tactile sensing capability. The sensor models and further information about the GelTip sensor can be found at http://danfergo.github.io/geltip.
BibTeX
@inproceedings{gomes2020ieee,
  author = {Daniel Fernandes Gomes and Zhonglin Lin and Shan Luo},
  title = {{2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)}},
  booktitle = {Openalex Unknown},
  year = {2020},
  doi = {10.1109/iros45743.2020},
}

Loading related content...

Figure