A Deep-Learning Based System for Rapid Genus Identification of Pathogens under Hyperspectral Microscopic Images

Chenglong Tao, Jian Du, Yingxin Tang, Jun‐Jie Wang, Ke Dong, Ming–Hsuan Yang, Bingliang Hu, Zhoufeng ZhangCells 2022 · DOI

TL;DR

This paper introduces an AI-powered system that combines hyperspectral microscopy with a novel deep learning architecture, "Buffer Net," to rapidly identify bacterial genera from single-cell images. Achieving 94.9% accuracy, this system offers a significant improvement over traditional methods, enabling faster, cheaper, and more accessible pathogen identification.

What Problem Does This Paper Address?

Infectious diseases pose a significant global health threat, and their effective management hinges on rapid and accurate identification of the causative pathogens. Current methods for bacterial identification, such as morphological examination, PCR, and MALDI-TOF, are often time-consuming, expensive, require specialized expertise, or involve laborious culturing steps. This delay in diagnosis can lead to suboptimal treatment, increased healthcare costs, and the potential for worsening patient conditions. The paper aims to bridge this gap by developing an automated, low-cost, and rapid system for identifying pathogenic bacteria at the genus level directly from microscopic images, eliminating the need for lengthy culturing processes.

Methods

The core of this work lies in the integration of hyperspectral microscopic imaging (HMI) with a custom-designed deep learning model called "Buffer Net."

  • Hyperspectral Microscopic Imaging (HMI): The researchers developed a new HMI system with a broader wavelength range (440–1023 nm) and finer spectral resolution (2.1 nm) compared to previous systems. This system captures "datacubes" of individual bacteria, containing both spatial and spectral information.
  • Data Preprocessing: To handle the HMI data, a series of preprocessing steps were implemented. This included discarding noisy spectral bands, performing flat-field correction for brightness uniformity, and using K-means clustering to segment bacteria from the background. Individual bacteria were then cropped from the datacubes.
  • Buffer Net Architecture: The proposed "Buffer Net" is a 3D Convolutional Neural Network (3D-CNN) designed to process the full hyperspectral datacubes without decomposing them. This preserves the crucial spatial-spectral correlations. The architecture features:
  • 3D Convolutional Layers: These layers extract features from the volumetric data.
  • Down-sampling Layers: Used to reduce the spatial dimensions of the feature maps.
  • Buffering Layers: These layers have a stride of 1, allowing for enhanced feature representation without reducing resolution, which is crucial for small bacterial objects.
  • Block Structure: The network is organized into blocks, each containing a down-sampling layer followed by buffering layers. This hierarchical structure allows for learning features at different scales.
  • SoftMax Output: The final layer provides probabilities for each bacterial genus.

The system integrates the HMI hardware, the Buffer Net algorithm, and interactive software for a complete workflow from imaging to diagnosis reporting.

Datasets

The study utilized a newly constructed, large-scale dataset comprising over 130,000 hyperspectral microscopic images of bacteria from 11 different genera. These genera were selected from both the Second Affiliated Hospital of Air Force Military Medical University and the American Type Culture Collection (ATCC) to ensure diversity. The dataset was split into training (70%) and testing (30%) sets. This dataset is significantly larger and has a broader spectral range than those used in previous hyperspectral imaging studies for bacteria identification.

Key Insights and Ideas

  • Preserving Spatial-Spectral Correlation is Key: The success of Buffer Net highlights the importance of processing hyperspectral datacubes holistically. Unlike methods that decompose or average spectral information, Buffer Net's 3D-CNN approach effectively leverages the combined spatial and spectral signatures of bacteria.
  • Buffering Layers Enhance Feature Representation: The ablation study, where buffering layers were removed, demonstrated their critical role. These layers, by maintaining resolution while extracting features, significantly improved the model's accuracy, suggesting they are vital for capturing subtle bacterial characteristics.
  • Single-Cell Identification is Feasible and Beneficial: The system's ability to identify bacteria at the single-cell level bypasses the need for culturing, drastically reducing identification time and complexity. This is a major step towards point-of-care diagnostics.
  • AI-Assisted Diagnosis for Novices: The user-friendly interface and automated nature of the system make it accessible to clinicians and technicians without extensive specialized training in microscopy or AI.

Key Results

  • High Accuracy: The Buffer Net achieved an impressive overall accuracy of 94.9% in differentiating between 11 bacterial genera. This outperformed other evaluated algorithms, including 1D-CNN (82.6%), 2D-CNN (91.3%), and 3D-ResNet (92.3%).
  • Rapid Identification: The entire AI-assisted system can perform imaging, data preprocessing, and genus identification in approximately 60 seconds, with the identification step taking only 30 seconds per hyperspectral microscopic image.
  • Improved Spectral Range and Resolution: The HMI system's wider spectral range (440–1023 nm) and finer spectral resolution (2.1 nm) provide richer data for classification compared to prior studies.
  • High Precision for Most Genera: Buffer Net achieved high differentiation precisions for most genera, with some reaching as high as 98% (e.g., Pseudomonas, Escherichia).

Limitations and Future Work

The authors acknowledge several limitations and propose future directions:

  • Lower Precision for Similar Genera: The precision for differentiating certain genera (Serratia, Klebsiella, Acinetobacter, and Stenotrophomonas) was relatively lower, likely due to their inherent similarities. Future work will focus on optimizing these classifications.
  • Genus-Level vs. Species/Subspecies Identification: The current system identifies bacteria at the genus level. Future research will explore extending this to species and subspecies identification, which requires even finer discrimination of spectral and spatial features.
  • Dataset Augmentation: While the dataset of over 130,000 images is substantial, the authors recognize the need to further augment the number and diversity of genera in the training and testing data to cover a wider range of pathogens encountered in routine clinical practice.

Summary updated

Abstract

Infectious diseases have always been a major threat to the survival of humanity. Additionally, they bring an enormous economic burden to society. The conventional methods for bacteria identification are expensive, time-consuming and laborious. Therefore, it is of great importance to automatically rapidly identify pathogenic bacteria in a short time. Here, we constructed an AI-assisted system for automating rapid bacteria genus identification, combining the hyperspectral microscopic technology and a deep-learning-based algorithm Buffer Net. After being trained and validated in the self-built dataset, which consists of 11 genera with over 130,000 hyperspectral images, the accuracy of the algorithm could achieve 94.9%, which outperformed 1D-CNN, 2D-CNN and 3D-ResNet. The AI-assisted system we developed has great potential in assisting clinicians in identifying pathogenic bacteria at the single-cell level with high accuracy in a cheap, rapid and automatic way. Since the AI-assisted system can identify the pathogenic genus rapidly (about 30 s per hyperspectral microscopic image) at the single-cell level, it can shorten the time or even eliminate the demand for cultivating. Additionally, the system is user-friendly for novices.
BibTeX
@inproceedings{tao2022deep,
  author = {Chenglong Tao and Jian Du and Yingxin Tang and Jun‐Jie Wang and Ke Dong and Ming–Hsuan Yang and Bingliang Hu and Zhoufeng Zhang},
  title = {{A Deep-Learning Based System for Rapid Genus Identification of Pathogens under Hyperspectral Microscopic Images}},
  booktitle = {Cells},
  year = {2022},
  doi = {10.3390/cells11142237},
}
Figure