Introduction
In Robot-Assisted Minimally Invasive Surgery (RMIS), the objective evaluation of surgical skill is paramount for effective training and patient safety. Currently, this process relies heavily on manual Global Rating Scales (GRS) such as the Objective Structured Assessment of Technical Skills (OSATS). However, manual assessment is inherently slow, subjective, and resource-intensive.
Automating this process using artificial intelligence presents a significant challenge. Surgical videos often exhibit subtle differences; a high-performing surgeon and a novice might perform the same task with visually similar movements, yet differ significantly in efficiency, tissue handling, and smoothness. Standard neural networks often struggle with this "needle in a haystack" problem when performing direct regression—predicting a single score from a single video. The authors of this work argue that direct regression's inherent dimensionality reduction fails to capture the nuanced representations of surgical skill.
Methodology: Contrastive Regression
To address these limitations, the authors propose Contra-Sformer, a novel architecture that shifts the paradigm from absolute to contrastive regression. Instead of evaluating a performance in isolation, the model compares a trainee's "test video" against a "gold standard" reference video—the highest-scoring execution in the dataset.
As illustrated in the architecture overview, the system consists of three primary stages:
- Feature Extraction: A dual-stream encoder processes both the input and reference videos using shared weights.
- Action-Aware Transformer: A Multi-Head Attention mechanism treats the test video as the "query" (q) and the reference video as the "keys" (k) and "values" (v), explicitly identifying temporal deviations.
- Relative Scoring: The model predicts a relative score ($Δs$), which is added to the known reference score ($s_{ref}$) to produce the final predicted score ($ŝ$).
Spatio-Temporal Feature Extraction
The feature extraction stage is critical for capturing both the visual details of surgical gestures and their temporal progression.
The system utilizes a ResNet-18 spatial encoder fine-tuned on surgical gestures to generate spatial feature matrices ($f^{(0)}$). These are then passed through a customized Temporal Convolutional Network (TCN) consisting of multiple layers ($f^{(1)}$ to $f^{(4)}$). This hierarchical approach allows the model to encode "gesture-aware" features that represent the surgery's progression over time before they are compared in the transformer module.
Experimental Evaluation
The Contra-Sformer was evaluated using the JIGSAWS (JHU-ISI Gesture and Skill Assessment Working Set) dataset, a standard benchmark in robotic surgery. The dataset includes video and kinematic data for three fundamental tasks: Suturing (SU), Needle Passing (NP), and Knot Tying (KT). Trials were performed by surgeons across three expertise levels (Expert, Intermediate, and Novice), with ground-truth GRS scores ranging from 6 to 30.
Key Insights and Results
The "Keep Your Eye on the Best" approach yields several critical insights:
- The Power of Comparison: Comparing performance to the best possible execution mimics the traditional "expert-apprentice" medical training model and proves significantly more effective than absolute regression.
- Interpretability through Attention: By visualizing the transformer's attention weights, the authors found that the model naturally focuses on moments of sub-optimal performance, such as needle drops or inefficient grasping. This provides a layer of temporal interpretability, allowing the system to pinpoint exactly when mistakes occurred.
- Performance Metrics: Contra-Sformer achieved high Spearman’s Correlation Coefficients (SCC) between 0.65 and 0.89. The Normalized Mean Absolute Error (NMAE) remained low, between 5.8% and 13.4%. The model demonstrated particular strength in complex tasks like Knot Tying and Suturing, outperforming existing state-of-the-art methods like ViSA and C3D-MTL-VF.
Conclusion and Future Directions
Contra-Sformer represents a significant step forward in automated surgical skill assessment. By leveraging contrastive learning and transformer-based attention, it moves beyond "black box" scoring toward interpretable, reference-based evaluation.
Future research will focus on reducing the dependency on manual gesture labels for pre-training through self-supervised learning. Additionally, transitioning from "dry lab" datasets to real-world clinical video—with challenges like smoke, blood, and varying camera angles—will be essential for practical deployment in operating rooms.
Summary updated