Multimodal Sensing for Robot-Assisted Sub-Tissue Feature Detection in Physiotherapy Palpation

Hao Li, Tian-Ao Ren, Jorge Garcia, Seongheon Hong, Jared Grinberg, Hojung Choi, Julia Di, Hao Li 0076, Dmitry Grinberg, Mark R. CutkoskyarXiv 2025 · DOI · arXiv

Physiotherapy and clinical examinations often require a practitioner to apply significant pressure while simultaneously detecting subtle subsurface features, such as tendons or muscle knots. Replicating this human capability in robotics is a significant challenge. While traditional robotic palpation relies on force-torque sensors, these sensors often struggle with the ambiguity of soft tissue interactions. Conversely, high-resolution vision-based tactile sensors are typically too delicate for the high-load environments (often exceeding 100 N) required for therapeutic massage.

This paper introduces PhysioVisionFT (PVFT), a novel multimodal sensor designed to bridge this gap. By integrating a ruggedized optical tactile dome with a compact 6-axis force-torque sensor, the PVFT system enables robots to "see" subsurface geometries while maintaining the precise force control necessary for safe and effective physiotherapy.

The PhysioVisionFT Sensor Architecture

The PVFT sensor is a hybrid device comprising two primary sensing modalities. The first is a vision-based tactile sensor consisting of a fisheye camera directed toward an internal, LED-illuminated polyurethane dome. To withstand high contact loads, the dome features a specialized three-layer construction: a stiff transparent core, a reflective mirror-effect layer, and a semi-opaque diffusion layer. This design ensures that the sensor remains robust under pressure while providing clear visual feedback of surface deformations.

The second component is a "CoinFT" sensor—a compact, 6-axis force-torque sensor mounted directly beneath the optical assembly. To ensure accuracy under high-load conditions, the researchers utilized a 5-layer Multilayer Perceptron (MLP) to calibrate the force data against a commercial ground-truth sensor. This multimodal approach allows the system to capture high-resolution spatial data alongside precise force vectors.

Experimental Methodology

To evaluate the PVFT sensor, the team mounted it on a Flexiv Rizon 4 robotic arm. The experiments focused on "ploughing" motions, where the robot pressed into a surface with 25–45 N of force and moved laterally. This motion mimics common massage techniques used to identify and treat musculoskeletal issues.

The testing environment utilized custom-cast silicone phantoms (Ecoflex 00-20) containing 3D-printed PLA tendons. These phantoms were designed to present various geometric challenges, including:

  • Presence vs. absence of tendons.
  • Variations in tendon depth (1.5 mm difference).
  • Complex geometries, such as crossing vs. straight tendons.
  • Single vs. multiple parallel tendons.

Results: Geometric Awareness and Force Tracking

The experimental results highlighted a critical limitation of traditional force sensing: it is effectively "blind" to subsurface geometry. When sliding over different tendon configurations, the force profiles generated by the 6-axis sensor were nearly identical. For example, the force signal could not reliably distinguish between a single thick tendon and two thin parallel ones.

In contrast, the vision-based tactile images provided clear, distinct signatures for each geometry. The deformation of the sensor's dome visually split into a "double-ridge" pattern when encountering two tendons, a feature that was easily classified. Using a ResNet-18 model to process these tactile images, the system achieved a perfect F1 score (1.00) for tendon detection. In comparison, relying solely on force data yielded an F1 score of only 0.78.

Beyond detection, the PVFT system demonstrated excellent force regulation. During high-pressure sliding tasks, the system maintained a commanded 25 N normal force with a Root Mean Square Error (RMSE) of just 7.04%. This confirms that the sensor can safely regulate therapeutic loads while simultaneously mapping the tissue structure.

Discussion and Future Directions

While the PVFT sensor represents a significant step forward, some limitations remain. The current study used rigid 3D-printed tendons in silicone phantoms, which are simplified models of heterogeneous human tissue. Furthermore, high-load "ploughing" can introduce large bending moments that cause slight drifts in force measurement, suggesting a need for even more robust calibration in future iterations.

Future work will focus on integrating the PVFT sensor into multimodal robot-learning frameworks, such as diffusion policies. By training on data from expert clinicians, the researchers aim to develop robots capable of closed-loop palpation—automatically adjusting their path and pressure based on the detected depth and geometry of a patient's tendons. Ultimately, this technology paves the way for autonomous robotic systems that can perform complex clinical exams and therapeutic massages with human-like sensitivity and safety.

Summary updated

Abstract

Robotic palpation relies on force sensing, but force signals in soft-tissue environments are variable and cannot reliably reveal subtle subsurface features. We present a compact multimodal sensor that integrates high-resolution vision-based tactile imaging with a 6-axis force-torque sensor. In experiments on silicone phantoms with diverse subsurface tendon geometries, force signals alone frequently produce ambiguous responses, while tactile images reveal clear structural differences in presence, diameter, depth, crossings, and multiplicity. Yet accurate force tracking remains essential for maintaining safe, consistent contact during physiotherapeutic interaction. Preliminary results show that combining tactile and force modalities enables robust subsurface feature detection and controlled robotic palpation.
BibTeX
@misc{li2025multimodal,
  author = {Hao Li and Tian-Ao Ren and Jorge Garcia and Seongheon Hong and Jared Grinberg and Hojung Choi and Julia Di and Hao Li and Dmitry Grinberg and Mark R. Cutkosky},
  title = {{Multimodal Sensing for Robot-Assisted Sub-Tissue Feature Detection in Physiotherapy Palpation}},
  year = {2025},
  eprint = {2512.20992},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2512.20992},
}
Figure