Using deep learning to identify the recurrent laryngeal nerve during thyroidectomy

Julia Gong, F. Christopher Holsinger, Julia E. Noel, Sohei Mitani, Jeffrey K. Jopling, Nikita Bedi, Yoon Woo Koh, Lisa A. Orloff, Cláudio Roberto Cernea, Serena YeungScientific 2021 · DOI

TL;DR

This paper introduces an end-to-end deep learning system for automatically identifying and measuring the recurrent laryngeal nerve (RLN) during thyroidectomies using standard handheld cameras. The system demonstrates promising performance, especially under optimal image capture conditions, paving the way for AI-assisted surgical guidance in open surgery.

What Problem Does This Paper Address?

Surgeons performing thyroidectomies must meticulously identify and avoid damaging the recurrent laryngeal nerve (RLN), crucial for speech. This visual discrimination is challenging, particularly in the complex and variable environment of open surgery, where standardized imaging is less common than in other medical specialties. While AI has made strides in fields like radiology and pathology, its application in open surgery for tasks like nerve identification has lagged. This gap is exacerbated by the unstructured nature of operating room images captured by handheld cameras, which often suffer from varying lighting, distance, and clutter. The paper aims to bridge this gap by developing a robust deep learning tool to assist surgeons in this critical identification task.

Methods

The researchers developed an end-to-end pipeline for joint nerve segmentation and measurement. The pipeline consists of two main stages:

  1. Image Cropping: An initial convolutional neural network (ResNeXt50) is used to automatically crop images, focusing on the surgical wound region. This step is crucial for removing irrelevant clutter and preparing the image for nerve segmentation.
  2. Nerve Segmentation and Measurement: A Mask R-CNN model is employed for segmenting the RLN. This model is trained using a multi-task loss function that includes binary cross-entropy for mask prediction and classification, and smooth L1 loss for bounding box localization. For nerve width estimation, the pipeline leverages the segmented nerve and a simultaneously segmented army-navy retractor (a surgical tool with a known 15mm width) to calibrate pixel measurements to real-world units.

Data augmentation techniques, including geometric and stylistic transformations, were extensively used to enhance model robustness against variations in image capture.

Datasets

The study utilized a newly curated dataset of 277 color (RGB) photographs taken during thyroidectomies from 130 patients between 2013 and 2020 at Stanford University. These images were collected using digital SLRs and smartphones, representing a diverse range of lighting conditions (bright, medium, dim) and picture distances (far-away, close-up). The dataset includes manual annotations for the RLN and, for some images, surgical retractors, performed by head and neck endocrine surgeons.

Key Insights and Ideas

  • Open Surgery's Unique Challenges: The paper highlights that open surgery presents significantly more complex and variable image capture conditions compared to endoscopic or laparoscopic procedures, making AI application more challenging.
  • Importance of Image Quality: The study strongly emphasizes that image capture quality (specifically, close-up distance and medium lighting) is a critical determinant of the deep learning model's performance.
  • End-to-End Pipeline for Practicality: Developing a complete pipeline from image cropping to nerve measurement makes the tool more practical for real-world surgical use.
  • AI as a Surgical Augmentation Tool: The work positions AI not as a replacement for surgeons but as a valuable tool to augment their judgment and improve outcomes, especially for less experienced surgeons.

Key Results

  • Segmentation Performance: The nerve segmentation model achieved a Dice Similarity Coefficient (DSC) of 0.707 (±0.075) under optimal conditions (close-up distance and medium lighting). Across all image conditions, DSC scores ranged from 0.343 (±0.077) to 0.707 (±0.075).
  • Cropping Model Performance: The cropping model achieved an average precision (AP) of 0.756, with better performance for close-up images (0.872 AP) than far-away images (0.677 AP).
  • Nerve Width Estimation: The nerve width estimation method, calibrated using a retractor, produced estimates that closely approximated ground-truth measurements, with differences less than 1 mm. The measured widths (1-4 mm) fall within previously published ranges for RLN width.
  • Qualitative Findings: Visual examples demonstrate that the model can successfully segment nerves even in challenging scenarios like thin or small nerves, but struggles with poor lighting, significant clutter, or irregular nerve shapes.

Limitations and Future Work

  • Dataset Size: While considered one of the largest for this specific task, the dataset of 277 images is relatively small for supervised deep learning.
  • Segmentation Precision: The algorithm doesn't always capture the precise border of nerves, especially smaller ones, those less exposed, or those with irregular shapes or environmental distractors.
  • Image Capture Impact: The significant variation in performance across different image conditions underscores the need for standardized image capture techniques.

Future work could involve:

  • Incorporating additional data augmentation methods.
  • Integrating medical prior knowledge into the model.
  • Improving nerve width estimation by accounting for camera and object pose.
  • Collecting data across all stages of the operation for real-time recognition.
  • Exploring augmented, mixed, and extended reality applications.

Summary updated

Abstract

Surgeons must visually distinguish soft-tissues, such as nerves, from surrounding anatomy to prevent complications and optimize patient outcomes. An accurate nerve segmentation and analysis tool could provide useful insight for surgical decision-making. Here, we present an end-to-end, automatic deep learning computer vision algorithm to segment and measure nerves. Unlike traditional medical imaging, our unconstrained setup with accessible handheld digital cameras, along with the unstructured open surgery scene, makes this task uniquely challenging. We investigate one common procedure, thyroidectomy, during which surgeons must avoid damaging the recurrent laryngeal nerve (RLN), which is responsible for human speech. We evaluate our segmentation algorithm on a diverse dataset across varied and challenging settings of operating room image capture, and show strong segmentation performance in the optimal image capture condition. This work lays the foundation for future research in real-time tissue discrimination and integration of accessible, intelligent tools into open surgery to provide actionable insights.
BibTeX
@inproceedings{gong2021using,
  author = {Julia Gong and F. Christopher Holsinger and Julia E. Noel and Sohei Mitani and Jeffrey K. Jopling and Nikita Bedi and Yoon Woo Koh and Lisa A. Orloff and Cláudio Roberto Cernea and Serena Yeung},
  title = {{Using deep learning to identify the recurrent laryngeal nerve during thyroidectomy}},
  booktitle = {Scientific},
  year = {2021},
  doi = {10.1038/s41598-021-93202-y},
}
Figure