SATac: A Thermoluminescence Enabled Tactile Sensor for Concurrent Perception of Temperature, Pressure, and Shear

Ziwu Song, Ran Yu, Xuan Zhang, Kit Wa Sou, Shilong Mu, Dengfeng Peng, Xiao-Ping Zhang, Wenbo Ding

TL;DR

This paper introduces SATac, a novel vision-based tactile sensor that can simultaneously perceive temperature, pressure, and shear. By leveraging the thermoluminescence properties of strontium aluminate (SA) and analyzing marker displacements with Voronoi diagrams, SATac offers a more comprehensive tactile perception for robots, enabling safer and more nuanced interactions with their environment.

What Problem Does This Paper Address?

Current robotic tactile sensors often rely on elastomer deformation to infer information like pressure and shear. However, a critical aspect of human tactile perception – temperature – is frequently overlooked or addressed by separate, less integrated systems. This limitation hinders robots from achieving truly human-like dexterity and safety, especially in tasks requiring nuanced interaction with objects of varying temperatures or where thermal cues are vital for avoiding damage. SATac aims to bridge this gap by providing a single, high-resolution sensor capable of concurrently measuring temperature alongside pressure and shear.

Methods

SATac's core innovation lies in its multimodal sensing approach:

  • Temperature Sensing: The sensor utilizes Strontium Aluminate (SA), a thermoluminescent material. When exposed to UV light (charging), SA emits visible light. The intensity of this emitted light is directly proportional to the temperature it has experienced. This allows SATac to map temperature distributions across its surface.
  • Pressure and Shear Sensing: A black marker array is embedded within the SA film. When the sensor deforms under pressure or shear, the relative positions and areas of the Voronoi polygons formed by these markers change. By analyzing these changes (specifically, area change rate for pressure and centroid displacement for shear), the sensor can infer pressure and shear forces.
  • Vision-Based Approach: A camera captures images of the SA film and its embedded markers. Sophisticated image processing techniques, including guided filtering, inpainting, and Voronoi diagram analysis, are employed to extract the temperature, pressure, and shear information from these images.
  • Dual-Light Field: SATac uses a UVC light field (275 nm) for charging the SA material and a visible fluorescence field (520 nm) as a backdrop for marker detection, ensuring clear imaging and efficient charging.

Datasets

Not discussed in the paper. The paper focuses on characterizing the sensor's performance through controlled experiments rather than training on large datasets.

Key Insights and Ideas

  • Unlocking Temperature Perception: The integration of thermoluminescence into a vision-based tactile sensor is a significant step towards richer tactile sensing. This allows robots to "feel" heat, a crucial but often neglected modality.
  • Synergistic Multimodality: SATac demonstrates that temperature, pressure, and shear can be perceived concurrently and with minimal interference from a single sensor. This is achieved through clever material selection and image processing.
  • High-Resolution Spatial Awareness: The vision-based approach, combined with the marker tracking, allows for high-resolution mapping of all three modalities across the sensor's surface.
  • Overcoming Environmental Challenges: The paper highlights SATac's potential in scenarios where conventional thermal imaging struggles, such as with reflective surfaces or hot air, due to its direct contact-based temperature sensing.

Key Results

  • SATac successfully demonstrates concurrent perception of temperature, pressure, and shear.
  • Temperature sensing shows a clear correlation between brightness and temperature in the range of 50°C to 180°C, with a narrow error band indicating stable performance.
  • Pressure and shear sensing exhibit a strong linear correlation with applied forces, with relatively small errors.
  • The sensor can discern object shapes and spatial positions of contact areas due to its high resolution.
  • Experiments show SATac's ability to handle complex shapes and varying contact conditions.

Limitations and Future Work

  • Temperature Range: The current temperature sensing range is limited to 50°C to 180°C, which is suitable for many everyday scenarios but could be expanded.
  • Future Work: The authors suggest exploring customization of the temperature sensing range through specific ion doping of the SA material to cover broader temperature spectrums. They also aim to further explore SATac's applications in various scenarios.

Summary updated

Abstract

Most vision-based tactile sensors use elastomer deformation to infer tactile information, which can not sense some modalities, like temperature. As an important part of human tactile perception, temperature sensing can help robots better interact with the environment. In this work, we propose a novel multi-modal vision-based tactile sensor, SATac, which can simultaneously perceive information on temperature, pressure, and shear. SATac utilizes the thermoluminescence of strontium aluminate to sense a wide range of temperatures with exceptional resolution. Additionally, the pressure and shear can also be perceived by analyzing the Voronoi diagram. A series of experiments are conducted to verify the performance of our proposed sensor. We also discuss the possible application scenarios and demonstrate how SATac could benefit robot perception capabilities.
BibTeX
@inproceedings{song2024satac,
  author = {Ziwu Song and Ran Yu and Xuan Zhang and Kit Wa Sou and Shilong Mu and Dengfeng Peng and Xiao-Ping Zhang and Wenbo Ding},
  title = {{SATac: A Thermoluminescence Enabled Tactile Sensor for Concurrent Perception of Temperature, Pressure, and Shear}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2024},
  url = {https://openreview.net/forum?id=sYQDcxRbg7},
}
Figure