Introduction
Laparoscopic cholecystectomy is a cornerstone of modern surgical practice. However, the increasing demand for surgical procedures, coupled with a global shortage of medical personnel, necessitates the development of collaborative surgical robots (cobots). Unlike teleoperated systems, cobots are designed to work alongside surgeons autonomously, requiring a high degree of context-awareness to anticipate and execute tasks. This study presents a fine-grained activity model based on "quadruplets" to enable such robotic collaboration.
Methodology: The Quadruplet Model
To provide robots with the necessary context, the researchers developed a model that extends beyond simple action triplets. The proposed quadruplet model consists of:
- Actor: Identifying the specific hand involved (e.g., Surgeon's left hand vs. Assistant's right hand).
- Instrument: The tool being utilized (e.g., Grasper, Clipper).
- Action: The specific verb or task (e.g., Hold, Cut, Cauterize).
- Target: The anatomical structure or object being manipulated (e.g., Gallbladder, Liver).
This granular approach allows for a precise mapping of surgical workflow, distinguishing between the roles of the primary surgeon and the assistant.
Data and Machine Learning
The study introduced the HeiCholeActivity dataset, comprising 20 annotated videos (11.7 hours) of both human and ex-vivo porcine procedures. This dataset is unique for its inclusion of "Actor" labels and its bridge between preclinical animal models and clinical human data. For activity recognition, the researchers employed Distilled-Swin, a Swin Transformer-based model. The model was trained on a combination of HeiCholeActivity and the benchmark CholecT45 dataset to improve generalizability and performance in the face of significant class imbalance.
Key Results and Automation Potential
The machine learning model achieved a mean Average Precision (mAP) of 37.9% and a Top-5 accuracy of 85.3%. While recognizing infrequent "edge case" activities remains a challenge, the model demonstrated robust performance on frequent surgical tasks.
A critical finding of the activity analysis is the quantification of robotic potential. The study revealed that up to 83.5% of an assistant's tissue-interacting tasks (excluding camera control) consist of repetitive "grasp" and "hold" actions. This suggests that even low-level robotic autonomy could significantly reduce the physical and cognitive workload of surgical assistants.
Conclusion
The development of a standardized quadruplet model and the HeiCholeActivity dataset marks a significant step toward autonomous surgical collaboration. By identifying that the vast majority of assistant tasks are candidates for automation, this research provides a roadmap for the deployment of cobots in the operating room. Future work will focus on integrating multi-modal data and scaling datasets to overcome current limitations in recognizing rare surgical events.
Summary updated