Abstract
Anomaly detection in Minimally-Invasive Surgery (MIS) traditionally requires a human expert monitoring the procedure from a console. Data scarcity, on the other hand, hinders what would be a desirable migration towards autonomous robotic-assisted surgical systems. Automated anomaly detection systems in this area typically rely on classical supervised learning. Anomalous events in a surgical setting, however, are rare, making it difficult to capture data to train a detection model in a supervised fashion. In this work we thus propose an unsupervised approach to anomaly detection for robotic-assisted surgery based on deep residual autoencoders. The idea is to make the autoencoder learn the 'normal' distribution of the data and detect abnormal events deviating from this distribution by measuring the reconstruction error. The model is trained and validated upon both the publicly available Cholec80 dataset, provided with extra annotation, and on a set of videos captured on procedures using artificial anatomies ('phantoms') produced as part of the Smart Autonomous Robotic Assistant Surgeon (SARAS) project. The system achieves recall and precision equal to 78.4%, 91.5%, respectively, on Cholec80 and of 95.6%, 88.1% on the SARAS phantom dataset. The end-to-end system was developed and deployed as part of the SARAS demonstration platform for real-time anomaly detection with a processing time of about 25 ms per frame.
Introduction
In Minimally-Invasive Surgery (MIS), surgeons rely entirely on a video feed from an endoscopic camera. If something goes wrong—the lens blurs, smoke from cauterization obscures the view, or unexpected bleeding occurs—the consequences can be dire. While autonomous robotic assistants like the Smart Autonomous Robotic Assistant Surgeon (SARAS) aim to support surgeons, they need a way to "know" when the situation has turned abnormal.
Traditional AI models based on supervised learning require thousands of labeled examples of every possible error to work effectively. However, surgical anomalies are thankfully rare, making it nearly impossible to collect enough data for a supervised model. This paper addresses this "data scarcity" problem by shifting to an unsupervised framework: teaching a machine what "normal" looks like so it can flag anything else as a potential threat.
Methodology: Deep Residual Autoencoders
The authors employ a Deep Residual Autoencoder (DAE). An autoencoder is a neural network designed to compress an input (an image frame) into a small "bottleneck" representation and then reconstruct the original image from that bottleneck. By training only on successful, clean procedures, the model becomes an expert on the status quo, making it naturally sensitive to any deviation.
Key technical features include:
- Residual Units: The architecture uses "skip connections" (similar to ResNet) to preserve fine details and prevent the "vanishing gradient" problem, ensuring accurate reconstruction of normal frames.
- Reconstruction Error as a Metric: The system calculates the Root Mean Square Error (RMSE) between the original frame and the reconstructed version. A high error typically indicates an anomaly.
- The Smoke Paradox and Dual-Thresholding: Interestingly, the authors found that while most anomalies increase reconstruction error, smoke actually decreases it due to pixel saturation. To handle this, they implemented both an upper and lower threshold to trigger alarms.
Datasets and Anomalies
The model was evaluated against several common surgical complications, including bleeding, lens blurring, and equipment displacement.
Training and validation were performed on two primary sources:
- Cholec80: A public dataset of 80 gallbladder removal videos, manually annotated by the authors for anomalous events.
- SARAS Phantom Dataset: A proprietary dataset featuring 18 video clips of robotic prostatectomies performed on high-fidelity, 3D-printed artificial anatomies ("phantoms").
Key Results and Performance
The system demonstrated high reliability across different surgical environments:
- Cholec80 Dataset: Achieved 78.4% recall and 91.5% precision. It was particularly effective at detecting smoke (100% recall).
- SARAS Phantom Dataset: Achieved 95.6% recall and 88.1% precision.
- Real-Time Deployment: The system processes video at roughly 40 frames per second (25ms/frame) on a standard consumer-grade GPU (NVIDIA GTX 1070), meeting the requirements for live surgical assistance.
Conclusion and Future Work
This unsupervised approach successfully bypasses the need for massive labeled datasets of rare accidents. While currently analyzing video frame-by-frame, future work may incorporate temporal data (3D feature extraction) to improve accuracy by accounting for motion sequences. The authors also propose exploring Generative Adversarial Networks (GANs) to create even more robust representations of "normal" surgical distributions.
Summary updated