Unsupervised Anomaly Detection for a Smart Autonomous Robotic Assistant Surgeon (SARAS) Using a Deep Residual Autoencoder

R. Dinesh Jackson Samuel, Fabio Cuzzolin

Abstract

Ensuring safety in robotic surgery requires the ability to detect unexpected events in real-time. This paper presents an unsupervised approach using a Deep Residual Autoencoder (DRAE) to identify surgical anomalies without the need for large, labeled datasets of rare failures. By training on normal surgical footage, the model learns to flag deviations—such as bleeding or equipment issues—via reconstruction error. The system demonstrates high accuracy on both clinical and phantom datasets, operating at real-time speeds suitable for clinical integration.

Introduction

In Minimally-Invasive Surgery (MIS), clear visualization is paramount. However, various factors can compromise the endoscopic feed, including lens blurring, smoke from cauterization, or sudden bleeding. As we progress toward Smart Autonomous Robotic Assistant Surgeons (SARAS), these systems must be capable of identifying such "anomalies" to ensure patient safety.

The primary challenge in developing these detection systems is the scarcity of data; rare surgical complications are difficult to capture in the volumes required for traditional supervised learning. This research addresses this by utilizing an unsupervised learning paradigm: the model is trained to recognize "normal" surgical workflows and identifies anomalies as deviations from this learned norm.

Common surgical anomalies detected by the system, including bleeding, smoke, lens blurring, and camera displacement.

Methodology: Deep Residual Autoencoder (DRAE)

The proposed system centers on a Deep Residual Autoencoder (DRAE) architecture designed for high-fidelity image reconstruction.

Architecture and Logic

The DRAE consists of an encoder that compresses input video frames into a low-dimensional latent space (bottleneck) and a decoder that reconstructs the frame from this representation. A key innovation is the integration of residual connections. The model employs "short" connections within convolutional blocks and "long" residual connections (shortcuts) between corresponding layers of the encoder and decoder. These connections facilitate the learning of identity mappings and mitigate the vanishing gradient problem, allowing for a deeper, more effective network.

The DRAE architecture featuring encoding/decoding layers, residual blocks, and long-range shortcuts to optimize reconstruction error.

Anomaly Detection Mechanism

The model is trained exclusively on "normal" surgical sequences. Consequently, it becomes proficient at reconstructing standard surgical scenes. When presented with an anomaly, the reconstruction quality drops significantly. This discrepancy is quantified using the Root Mean Square Error (RMSE). An anomaly alert is triggered when the RMSE exceeds a predefined threshold, typically set at the 95th percentile of the error observed during training on normal data.

Experimental Evaluation

The system was validated using two primary datasets:

  1. Cholec80: A public dataset of 80 gallbladder removals, manually annotated for anomalies like bleeding and smoke.
  2. SARAS Phantom Dataset: 18 video clips of radical prostatectomies performed on 3D-printed phantom organs, focusing on technical anomalies such as blocked views.

Results and Discussion

The DRAE demonstrated superior performance compared to standard autoencoders and existing benchmarks like "Ganomaly." On the Cholec80 dataset, the system achieved 78.4% recall and 91.5% precision, proving particularly adept at detecting smoke and camera displacement. Performance on the SARAS Phantom dataset was even higher, reaching 95.6% recall.

Crucially, the system maintains a processing time of 25ms per frame (40 FPS) on a standard GPU. This real-time capability is essential for integration into live surgical consoles, where latency could pose a risk to the patient.

Conclusion and Future Work

This research highlights the potential of unsupervised learning for monitoring autonomous surgical assistants. By focusing on reconstruction error, the DRAE provides a scalable solution for detecting unpredictable events in the operating room. Future improvements will focus on localized error detection to identify small-scale anomalies and the incorporation of temporal features to better capture motion-based irregularities.

Summary updated

Abstract

Anomaly detection in Minimally-Invasive Surgery (MIS) traditionally requires a human expert monitoring the procedure from a console, whereas automated anomaly detection systems in this area typically rely on classical supervised learning. Anomalous surgical events, however, are rare, making it difficult to capture data to train a model in a supervised fashion. In this work we propose an unsupervised approach to anomaly detection for robotic MIS based on deep residual autoencoders. The idea is to make the autoencoder learn the `normal' distribution of the data and detect abnormal events deviating from this distribution by measuring a reconstruction error. The model is trained and validated upon both the publicly available Cholec80 dataset and a set of videos captured on procedures using artificial anatomies (`phantoms') as part of the Smart Autonomous Robotic Assistant Surgeon (SARAS) project. The system achieves recall and precision equal to 78.4%, 91.5%, respectively, on Cholec80 and of 95.6%, 88.1% on the SARAS phantom dataset. The system was developed and deployed as part of the SARAS platform for real-time anomaly detection with a processing time of 25 ms per frame.
BibTeX
@article{samuel2021unsupervised,
  author = {R. Dinesh Jackson Samuel and Fabio Cuzzolin},
  title = {{Unsupervised Anomaly Detection for a Smart Autonomous Robotic Assistant Surgeon (SARAS) Using a Deep Residual Autoencoder}},
  journal = {IEEE Robotics Autom. Lett.},
  year = {2021},
  doi = {10.1109/lra.2021.3097244},
}
Figure