A Deep Learning Approach to Classify Surgical Skill in Microsurgery Using Force Data from a Novel Sensorised Surgical Glove

Jialang Xu, Dimitrios Anastasiou, James Booker, Oliver E. Burton, Hugo Layard Horsfall, Carmen Salvadores Fernandez, Yang Xue, Danail Stoyanov, Manish K. Tiwari, Hani J. Marcus, Evangelos B. Mazomenos2023 · DOI

TL;DR

This paper introduces a novel approach to automatically assess surgical skill in microsurgery by analyzing force data captured by a sensorized surgical glove. Using deep learning models, they achieve high accuracy in distinguishing between expert and novice surgeons, paving the way for objective, real-time feedback in surgical training.

What Problem Does This Paper Address?

Microsurgery, a critical component of many surgical fields, has a steep learning curve. Traditionally, surgical skill assessment relies on subjective, time-consuming, and often biased apprenticeship models. While simulators and retrospective analysis tools exist, they lack real-time feedback capabilities. Errors in microsurgery are frequently linked to the inappropriate application of force, making force data a promising indicator of skill. However, existing methods for analyzing this data are often computationally intensive and don't provide immediate feedback. This paper addresses the need for an automated, objective, and real-time system to classify surgical skill in microsurgery using force data.

Methods

The researchers explored six deep learning architectures for surgical skill classification: Long Short-Term Memory (LSTM), Gate Recurrent Unit (GRU), Bidirectional LSTMs (Bi-LSTM), Convolutional Long Short-Term Memory Deep Neural Network (CLDNN), Temporal Convolutional Network (TCN), and Transformer. To enhance model performance and robustness, they also investigated six data augmentation techniques: Fast Fourier Transform (FFT), Quantization, Drift, Time Warp, Gaussian Noise, and Temporal Jittering. The study employed two cross-validation strategies: random split and Leave-One-User-Out (LOUO) to evaluate model generalizability. Performance was assessed using standard metrics like accuracy, precision, recall, and F1-score.

Datasets

The study introduced a novel dataset comprising 236 trials from 13 surgeons (categorized as 'Novice' or 'Expert') performing a validated preclinical microsurgical task called 'Star's the limit'. This task involved dissecting a star shape from a grape skin using micro-scissors and forceps, with force data collected from a sensorized surgical glove.

Key Insights and Ideas

  • Force Data is Highly Informative: The study strongly validates that force data captured by a sensorized glove contains rich information for distinguishing between expert and novice surgical skills.
  • Deep Learning Excels: Deep learning models, particularly TCN and CLDNN, demonstrate remarkable ability to learn complex patterns from force data for skill classification.
  • Data Augmentation is Crucial: While models perform well without it, data augmentation techniques like Time Warp and FFT significantly improve robustness and reduce performance variability, especially in more challenging evaluation settings like LOUO.
  • LOUO is a Stricter Test: The Leave-One-User-Out (LOUO) cross-validation scheme proved to be a much more challenging evaluation, highlighting the models' ability to generalize to unseen surgeons. CLDNN showed particular strength in this setting.
  • Qualitative Insights: Visualizations of model attention revealed that models tend to focus on areas with sharp force peaks and sudden changes for novice surgeons, while experts' executions showed more consistent, lower force application, which the models also learned to identify.

Key Results

  • Top Performance: The Temporal Convolutional Network (TCN) achieved the highest accuracy of 97.45% in the random split setting, while the Convolutional Long Short-Term Memory Deep Neural Network (CLDNN) achieved 96.19% in the LOUO setting.
  • Time Warp Augmentation: TCN with Time Warp data augmentation reached 97.46% accuracy in the random split, and CLDNN with Time Warp achieved 96.48% accuracy in the LOUO setting.
  • Model Comparison: TCN and CLDNN consistently outperformed other models across various metrics and cross-validation schemes.
  • LOUO Challenges: In the LOUO scheme, models showed higher False Negative rates, indicating a tendency to misclassify expert surgeons as novices, a common challenge in generalization tasks.

Limitations and Future Work

  • Sensor Placement and Customization: The current sensor placement on the glove might not be optimal for all instruments and surgical techniques. Future work will focus on optimizing placement and developing customizable gloves for different hand orientations and potentially wireless sensor integration.
  • Force Range and Haptic Feedback: For highly complex surgeries, the current sensor might not capture the full range of forces. Additionally, the impact of the sensor on the surgeon's haptic feedback needs further investigation.
  • Task Fidelity: The grape dissection task, while useful, is a low-fidelity simulation and doesn't fully replicate the pressure and complications of real-life surgery.
  • Dataset Size and Diversity: While 13 surgeons were recruited, the dataset is still considered limited. Future work should involve collecting data from more surgeons across different geographical locations and hospitals to increase data diversity and robustly validate the approaches.
  • Multimodal Data: The authors suggest that combining force data with other inputs, such as video, could lead to even more comprehensive and discriminative features for automated skill assessment.
  • Algorithm Limitations: The LOUO cross-validation highlighted challenges in generalization, with models showing higher variability and a tendency to misclassify experts as novices. The effectiveness of data augmentation techniques also varied, with Time Warp and FFT showing particular promise.

Summary updated

BibTeX
@misc{xu2023deep,
  author = {Jialang Xu and Dimitrios Anastasiou and James Booker and Oliver E. Burton and Hugo Layard Horsfall and Carmen Salvadores Fernandez and Yang Xue and Danail Stoyanov and Manish K. Tiwari and Hani J. Marcus and Evangelos B. Mazomenos},
  title = {{A Deep Learning Approach to Classify Surgical Skill in Microsurgery Using Force Data from a Novel Sensorised Surgical Glove}},
  year = {2023},
  doi = {10.3390/s23218947},
}
Figure