Screening adequacy of unstained thyroid fine needle aspiration samples using a deep learning-based classifier

Junbong Jang, Young H. Kim, Brian Westgate, Yang Zong, Caleb Hallinan, Ali Akalin, Kwonmoo LeeScientific 2023 · DOI

TL;DR

This paper introduces FNA-Net, a deep learning system that can assess the adequacy of unstained thyroid fine needle aspiration (FNA) samples directly from a glass slide. By combining two deep learning models, FNA-Net aims to reduce the need for repeat biopsies, saving time and cost for patients and healthcare systems.

What Problem Does This Paper Address?

Thyroid fine needle aspiration (FNA) is a common and effective diagnostic tool for thyroid nodules. However, a significant portion (around 10%) of initial FNA samples are deemed "non-diagnostic," meaning they don't contain enough cellular material or are of poor quality for accurate analysis. This necessitates repeat FNA procedures, leading to delays in diagnosis, increased patient anxiety, and higher healthcare costs. Traditionally, assessing sample adequacy is done through on-site evaluation (ROSE/ROSAA), which requires time-consuming staining and the presence of a cytopathologist. This paper tackles the challenge of automating this adequacy assessment process, specifically for unstained samples, to make it faster, cheaper, and more accessible.

Methods

The core of the solution is a deep learning model called FNA-Net. It's an ensemble approach that combines two distinct deep learning models:

  • Patch-based Whole Slide Image Classifier (MTL Classifier): This model uses a multi-task learning (MTL) framework. It takes small patches (256x256 pixels) of the unstained FNA slide image and is trained on multiple tasks simultaneously: classifying follicular clusters, estimating their size, reconstructing the image, and segmenting them. The MTL approach aims to improve classification performance by learning shared representations across these tasks.
  • Faster R-CNN: This is a well-established object detection model used for identifying and localizing follicular clusters within the whole slide image without prior patching.

FNA-Net's pipeline works in two stages:

  1. Follicular Cluster Detection: Both the MTL classifier and Faster R-CNN independently detect follicular clusters in the unstained FNA images.
  2. Adequacy Classification: The detection results from both models are combined (overlapped). The system then counts the total number of detected follicular clusters. If this count falls below a predetermined threshold, the slide is classified as non-diagnostic.

The authors also employed hierarchical bootstrapping to simulate a larger dataset for robust evaluation, addressing the limitation of a small initial test set.

Datasets

The study used FNA biopsy samples from six patients, resulting in a total of 21 unstained and stained slides. Ground truth labels for follicular clusters in unstained images were provided by pathologists. For evaluation, the dataset was split into training, validation, and test sets. To overcome the limited size of the test set, hierarchical bootstrapping was used to generate 10,000 simulated slides for more comprehensive performance evaluation.

Key Insights and Ideas

  • Unstained Sample Adequacy Assessment is Feasible: The most significant insight is that it's possible to accurately assess the adequacy of FNA samples using deep learning on unstained slides, bypassing the need for immediate staining and expert interpretation at the point of care.
  • Ensemble Approach for Robustness: Combining two different deep learning architectures (patch-based MTL and whole-slide Faster R-CNN) proved more effective than either model alone. This ensemble approach helps to mitigate the individual weaknesses of each model, leading to improved precision and IOU.
  • MTL for Feature Learning: The multi-task learning approach for the patch-based classifier was crucial for learning rich feature representations, even with limited data.
  • Precision vs. Recall Trade-off: The study highlights the importance of tuning the classification threshold to balance precision and recall, allowing clinicians to prioritize avoiding false negatives (missing inadequate samples) or false positives (incorrectly flagging adequate samples as inadequate), depending on clinical needs.

Key Results

  • FNA-Net Performance: FNA-Net achieved a 0.81 F1 score and 0.84 AUC in the precision-recall curve for detecting non-diagnostic slides (defined as having fewer than six follicular clusters) using bootstrapped samples.
  • Improved Precision: FNA-Net significantly improved precision over Faster R-CNN by combining the models, effectively reducing false positives.
  • Reduced False Positives: The ensemble approach reduced the number of false positives by half compared to Faster R-CNN alone, while maintaining similar true positive and false negative rates.
  • High Recall Preferred: Clinically, a high recall (minimizing false negatives) is often preferred, even at the cost of lower precision, to avoid missing inadequate samples and requiring repeat procedures. The system allows for user-adjustable thresholds to achieve this balance.

Limitations and Future Work

  • Limited Dataset Size: The study acknowledges the limitation of a small initial test set and relied heavily on bootstrapped simulation for evaluation.
  • Higher Magnification: The authors suggest that using higher magnification (e.g., 100X) could improve the accuracy of follicular cluster detection, though this would increase image acquisition time.
  • Image-to-Image Translation: To potentially leverage stained image information without actual staining, they propose using image-to-image translation techniques to "virtually stain" unstained images for training.
  • Prospective Clinical Validation: The authors emphasize the need for training with more extensive datasets and conducting prospective studies to validate FNA-Net in real clinical settings.
  • Generalizability: While the current focus is on thyroid FNA, the authors suggest that FNA-Net could be adapted for other FNA sample types by providing different labels for training.

Summary updated

Abstract

Fine needle aspiration (FNA) biopsy of thyroid nodules is a safe, cost-effective, and accurate diagnostic method for detecting thyroid cancer. However, about 10% of initial FNA biopsy samples from patients are non-diagnostic and require repeated FNA, which delays the diagnosis and appropriate care. On-site evaluation of the FNA sample can be performed to filter out non-diagnostic FNA samples. Unfortunately, it involves a time-consuming staining process, and a cytopathologist has to be present at the time of FNA. To bypass the staining process and expert interpretation of FNA specimens at the clinics, we developed a deep learning-based ensemble model termed FNA-Net that allows in situ screening of adequacy of unstained thyroid FNA samples smeared on a glass slide which can decrease the non-diagnostic rate in thyroid FNA. FNA-Net combines two deep learning models, a patch-based whole slide image classifier and Faster R-CNN, to detect follicular clusters with high precision. Then, FNA-Net classifies sample slides to be non-diagnostic if the total number of detected follicular clusters is less than a predetermined threshold. With bootstrapped sampling, FNA-Net achieved a 0.81 F1 score and 0.84 AUC in the precision-recall curve for detecting the non-diagnostic slides whose follicular clusters are less than six. We expect that FNA-Net can dramatically reduce the diagnostic cost associated with FNA biopsy and improve the quality of patient care.
BibTeX
@inproceedings{jang2023screening,
  author = {Junbong Jang and Young H. Kim and Brian Westgate and Yang Zong and Caleb Hallinan and Ali Akalin and Kwonmoo Lee},
  title = {{Screening adequacy of unstained thyroid fine needle aspiration samples using a deep learning-based classifier}},
  booktitle = {Scientific},
  year = {2023},
  doi = {10.1038/s41598-023-40652-1},
}
Figure