A Novel Framework for the Automated Characterization of Gram-Stained Blood Culture Slides Using a Large-Scale Vision Transformer

Jack McMahon, Naofumi Tomita, Elizabeth S. Tatishev, Adrienne A. Workman, Cristina R. Costales, Niaz Banaei, Isabella W. Martin, Saeed HassanpourarXiv 2024 · arXiv · OpenReview

Abstract

This study introduces a new framework for the artificial intelligence-assisted characterization of Gram-stained whole-slide images (WSIs). As a test for the diagnosis of bloodstream infections, Gram stains provide critical early data to inform patient treatment. Rapid and reliable analysis of Gram stains has been shown to be positively associated with better clinical outcomes, underscoring the need for improved tools to automate Gram stain analysis. In this work, we developed a novel transformer-based model for Gram-stained WSI classification, which is more scalable to large datasets than previous convolutional neural network (CNN)-based methods as it does not require patch-level manual annotations. We also introduce a large Gram stain dataset from Dartmouth-Hitchcock Medical Center (Lebanon, New Hampshire, USA) to evaluate our model, exploring the classification of five major categories of Gram-stained WSIs: Gram-positive cocci in clusters, Gram-positive cocci in pairs/chains, Gram-positive rods, Gram-negative rods, and slides with no bacteria. Our model achieves a classification accuracy of 0.858 (95% CI: 0.805, 0.905) and an AUC of 0.952 (95% CI: 0.922, 0.976) using five-fold nested cross-validation on our 475-slide dataset, demonstrating the potential of large-scale transformer models for Gram stain classification. We further demonstrate the generalizability of our trained model, which achieves strong performance on external datasets without additional fine-tuning.

Introduction

When a patient has a suspected bloodstream infection (sepsis), every hour counts. The "gold standard" for initial diagnosis is the Gram stain—a century-old technique where a lab technician identifies bacterial morphology and staining properties under a microscope. However, this manual process is time-consuming and prone to human error, with error rates reaching up to 2.7%. While previous AI attempts used convolutional neural networks (CNNs), they were hindered by the "annotation tax"—the need for manual pixel-level labeling—and the computational difficulty of processing gigapixel whole-slide images (WSIs).

Methods: The GramViT Framework

We introduce GramViT, which leverages the LongViT architecture—a transformer designed specifically to handle very large images. Unlike previous models, GramViT was trained using weakly supervised learning, relying only on final laboratory reports as ground truth for entire slides.

Key technical features include:

  • Large-Region Processing: GramViT processes massive 4,096x4,096 pixel regions, allowing the model to capture the distribution of bacteria across the slide.
  • High Resolution: The model utilizes native 40x magnification, as down-sampling to 20x was found to significantly degrade performance by obscuring fine morphological details.
  • Transfer Learning: Pre-training on the TCGA (The Cancer Genome Atlas) dataset provided robust low-level visual features effective for microbiology.
  • Automated Quality Control: The pipeline filters out non-informative blank areas and blurry regions caused by microscope oil or focus issues.

Datasets and Morphological Classification

The primary evaluation used a new dataset of 475 Gram-stained slides from Dartmouth-Hitchcock Medical Center (DHMC). The model classifies five categories: Gram-positive cocci (clusters and chains), Gram-positive rods, Gram-negative rods, and "No Bacteria."

Overview of a Gram-stained whole-slide image and the four primary bacterial morphologies identified by the GramViT framework.

Key Results

GramViT achieved an accuracy of 0.858 and an AUC of 0.952 on the DHMC dataset, significantly outperforming the DeepSlide CNN framework (accuracy 0.771). The model was particularly effective at identifying Gram-negative rods (0.925 F1-score) and Gram-positive cocci in clusters (0.906 F1-score).

A major finding was the model's generalizability. Without additional fine-tuning, GramViT maintained high performance on external datasets from Stanford Health (AUC 0.865) and Medical Faculty Mannheim. Furthermore, the model demonstrated clinical utility by correctly identifying the bacterial class in 8 out of 11 slides that had been initially misclassified by human technicians.

Discussion

GramViT demonstrates that large-scale Vision Transformers can overcome the limitations of traditional CNNs in microbiology. By utilizing slide-level labels and processing large spatial contexts, the framework provides a scalable solution for automated blood culture analysis. Future work will focus on expanding the model's scope to include rarer morphologies like yeast and mixed infections, alongside developing visualization tools to enhance clinical interpretability.

Summary updated

Abstract

This study introduces a new framework for the artificial intelligence-assisted characterization of Gram-stained whole-slide images (WSIs). As a test for the diagnosis of bloodstream infections, Gram stains provide critical early data to inform patient treatment. Rapid and reliable analysis of Gram stains has been shown to be positively associated with better clinical outcomes, underscoring the need for improved tools to automate Gram stain analysis. In this work, we developed a novel transformer-based model for Gram-stained WSI classification, which is more scalable to large datasets than previous convolutional neural network (CNN) -based methods as it does not require patch-level manual annotations. We also introduce a large Gram stain dataset from Dartmouth-Hitchcock Medical Center (Lebanon, New Hampshire, USA) to evaluate our model, exploring the classification of five major categories of Gram-stained WSIs: Gram-positive cocci in clusters, Gram-positive cocci in pairs/chains, Gram-positive rods, Gram-negative rods, and slides with no bacteria. Our model achieves a classification accuracy of 0.858 (95% CI: 0.805, 0.905) and an AUC of 0.952 (95% CI: 0.922, 0.976) using five-fold nested cross-validation on our 475-slide dataset, demonstrating the potential of large-scale transformer models for Gram stain classification. We further demonstrate the generalizability of our trained model, which achieves strong performance on external datasets without additional fine-tuning.
BibTeX
@misc{mcmahon2024novel,
  author = {Jack McMahon and Naofumi Tomita and Elizabeth S. Tatishev and Adrienne A. Workman and Cristina R. Costales and Niaz Banaei and Isabella W. Martin and Saeed Hassanpour},
  title = {{A Novel Framework for the Automated Characterization of Gram-Stained Blood Culture Slides Using a Large-Scale Vision Transformer}},
  year = {2024},
  eprint = {2409.15546},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2409.15546},
}
Figure