Recurrent and Spiking Modeling of Sparse Surgical Kinematics

Neil Getty, Zixuan Zhou, Stephan Gruessner, Liaohai Chen, Fangfang Xia

Abstract

Robot-assisted minimally invasive surgery is improving surgeon performance and patient outcomes. This innovation is also turning what has been a subjective practice into motion sequences that can be precisely measured. A growing number of studies have used machine learning to analyze video and kinematic data captured from surgical robots. In these studies, models are typically trained on benchmark datasets for representative surgical tasks to assess surgeon skill levels. While they have shown that novices and experts can be accurately classified, it is not clear whether machine learning can separate highly proficient surgeons from one another, especially without video data. In this study, we explore the possibility of using only kinematic data to predict surgeons of similar skill levels. We focus on a new dataset created from surgical exercises on a simulation device for skill training. A simple, efficient encoding scheme was devised to encode kinematic sequences so that they were amenable to edge learning. We report that it is possible to identify surgical fellows receiving near perfect scores in the simulation exercises based on their motion characteristics alone. Further, our model could be converted to a spiking neural network to train and infer on the Nengo simulation framework with no loss in accuracy. Overall, this study suggests that building neuromorphic models from sparse motion features may be a potentially useful strategy for identifying surgeons and gestures with chips deployed on robotic systems to offer adaptive assistance during surgery and training with additional latency and privacy benefits.

Introduction

In robot-assisted surgery, assessing a surgeon's skill traditionally involves analyzing both video feeds and kinematic logs. Most existing research focuses on the binary classification of "novices" versus "experts." This paper addresses a more challenging problem: distinguishing between multiple highly proficient surgeons using only sparse kinematic data. By omitting video data, this approach enhances privacy, reduces latency, and facilitates "edge" processing directly on surgical robotic hardware.

Methods and Materials

The researchers introduced a new dataset from the University of Illinois at Chicago (UIC), featuring data from four surgical fellows of similar proficiency levels. The data was captured using a MIMIC FlexVR simulator across four tasks: Pick and Place, Peg Board, Ring and Rail, and Thread the Needle.

To process this data for neuromorphic computing, a Sparse Event Encoding method was used. Instead of raw positional data, the model calculates movement between timesteps and applies a threshold to generate binary "events" (spikes). These spikes were used to train Long Short-Term Memory (LSTM) and Convolutional Neural Networks (CNNs). Finally, these models were converted into Spiking Neural Networks (SNNs) using the Nengo simulation framework, allowing for energy-efficient execution on specialized hardware like the Intel Loihi chip.

Results

The LSTM model achieved an accuracy of 83.43% in identifying individual surgeons and 79.14% in predicting the specific task being performed. A key finding was the "lossless" conversion to SNNs; the spiking models performed nearly identically to their non-spiking counterparts, validating the sparse encoding strategy.

Confusion matrix for surgeon identification, demonstrating the model's ability to distinguish between four highly proficient fellows.

Discussion and Feature Importance

The results suggest that surgeons possess unique "motion signatures" akin to fingerprints. Feature importance analysis revealed that the "Pitch" and "Roll" of the left arm were the most critical features for classification. This likely stems from the complex rotations required of the non-dominant hand during intricate tasks like Thread the Needle.

Feature importance analysis showing the accuracy difference when specific kinematic features are removed, highlighting the significance of left-arm orientation.

Conclusion

This study demonstrates that neuromorphic models built from sparse motion features can effectively identify surgeons and gestures. This strategy offers significant benefits for real-time adaptive assistance and training. Future work will focus on domain adaptation to clinical robotic systems and deployment on physical neuromorphic chips to measure real-world power and latency improvements.

Summary updated

Abstract

Robot-assisted minimally invasive surgery is improving surgeon performance and patient outcomes. This innovation is also turning what has been a subjective practice into motion sequences that can be precisely measured. A growing number of studies have used machine learning to analyze video and kinematic data captured from surgical robots. In these studies, models are typically trained on benchmark datasets for representative surgical tasks to assess surgeon skill levels. While they have shown that novices and experts can be accurately classified, it is not clear whether machine learning can separate highly proficient surgeons from one another, especially without video data. In this study, we explore the possibility of using only kinematic data to predict surgeons of similar skill levels. We focus on a new dataset created from surgical exercises on a simulation device for skill training. A simple, efficient encoding scheme was devised to encode kinematic sequences so that they were amenable to edge learning. We report that it is possible to identify surgical fellows receiving near perfect scores in the simulation exercises based on their motion characteristics alone. Further, our model could be converted to a spiking neural network to train and infer on the Nengo simulation framework with no loss in accuracy. Overall, this study suggests that building neuromorphic models from sparse motion features may be a potentially useful strategy for identifying surgeons and gestures with chips deployed on robotic systems to offer adaptive assistance during surgery and training with additional latency and privacy benefits.
BibTeX
@inproceedings{getty2020recurrent,
  author = {Neil Getty and Zixuan Zhou and Stephan Gruessner and Liaohai Chen and Fangfang Xia},
  title = {{Recurrent and Spiking Modeling of Sparse Surgical Kinematics}},
  booktitle = {ICONS},
  year = {2020},
  url = {https://openreview.net/forum?id=B7GwKGaI2Ip},
}
Figure