Point of Care Image Analysis for COVID-19

Daniel Yaron, Daphna Keidar, Elisha Goldstein, Yair Shachar, Ayelet Blass, Oz Frank, Nir Schipper, Nogah Shabshin, Ahuva Grubstein, Dror Suhami, Naama R. Bogot, Chedva S. Weiss, Eyal Sela, Amiel A. Dror, Mordehay Vaturi, Federico Mento, Elena Torri, Riccardo Inchingolo, Andrea Smargiassi, Gino Soldati, Tiziano Perrone, Libertario Demi, Meirav Galun, Shai Bagon, Yishai M. Elyada, Yonina C. Eldar

TL;DR

This paper introduces deep learning models for rapid COVID-19 detection and severity grading using portable chest X-rays (CXRs) and lung ultrasounds (LUS). By leveraging novel pre-processing techniques and large, curated datasets, they achieve over 90% detection accuracy for COVID-19 from CXRs and improve severity grading from LUS, paving the way for more accessible point-of-care diagnostics.

What Problem Does This Paper Address?

The COVID-19 pandemic highlighted the urgent need for fast, accurate, and accessible diagnostic tools. While RT-PCR tests are the gold standard, they suffer from issues like cost, slow turnaround times, and variable sensitivity. Medical imaging, particularly chest CT, can detect COVID-19, but CT scanners are expensive, non-portable, and require extensive decontamination, making them unsuitable for point-of-care (POC) settings. Chest X-rays (CXRs) and lung ultrasounds (LUS) are more portable and cost-effective POC modalities, but identifying COVID-19 findings in these images can be challenging for human interpretation. This paper aims to bridge this gap by developing deep learning models that can reliably detect COVID-19 from CXRs and assess its severity using LUS, enabling faster and more widespread diagnostics.

Methods

The researchers employed a two-pronged approach for CXR and LUS analysis.

For CXR-based COVID-19 detection, they utilized a deep learning pipeline that involved:

  • Data Augmentation: Applying various transformations (brightness, contrast, rotation, etc.) to increase the robustness of the model.
  • Normalization: Standardizing image brightness and resizing to a uniform resolution.
  • Lung Segmentation: A crucial step where a U-Net model was used to generate a pixel-mask of the lungs. This segmentation mask was then added as an extra input channel to the main network, providing explicit information about lung regions.
  • Transfer Learning: Pre-trained deep neural networks (ResNet34, ResNet50, ResNet152, VGG16, and Chexpert) were fine-tuned on their collected CXR dataset.
  • Ensemble Model: Combining the predictions of multiple trained networks to improve overall performance.
  • Image Retrieval: Developing a method to find similar CXR images based on learned embeddings, useful for clinical comparison.

For LUS-based severity grading, the approach focused on addressing the challenges posed by different ultrasound probe types:

  • Image Rectification: A novel signal processing technique was developed to "rectify" images captured by convex probes. This process aligns sonographic artifacts (B-lines) that appear tilted in convex probe images, making them axis-aligned similar to linear probe images. This rectification was applied as an additional input channel.
  • Ensemble of ResNet-18 Networks: Two ResNet-18 networks were trained: one on original LUS frames and another on the rectified frames. These were then combined into an ensemble.

Datasets

  • CXR Dataset: A large dataset of 2426 CXRs was collected from 1384 patients across four hospitals in Israel. This included 360 positive COVID-19 cases and 1024 negative cases. The negative cases were sourced from before the pandemic to ensure they were not COVID-19 related.
  • LUS Dataset: The study utilized the ICLUS dataset, which contains 58,924 LUS frames, with 78% captured using convex probes. This dataset was used for training and evaluation, with a focus on comparing their method to existing work on this dataset.

Key Insights and Ideas

  • Lung Segmentation as an Auxiliary Channel: Adding a lung segmentation mask as an explicit input channel to the CXR detection network significantly boosted performance. This suggests that guiding the network's attention to relevant anatomical regions is highly beneficial.
  • The Power of "Rectification" for LUS: The novel image rectification technique for convex LUS probes is a key innovation. By making sonographic artifacts axis-aligned, it simplifies the task for standard convolutional networks, leading to improved severity grading without needing more complex architectures.
  • Ensemble Methods for Robustness: Combining multiple models (ensemble) consistently yielded better results than individual models for both CXR detection and LUS grading, highlighting the benefit of diverse perspectives.
  • POC Modalities are Viable: The success in achieving high performance with CXR and LUS demonstrates their potential as effective POC tools for COVID-19 management, complementing or even replacing more cumbersome methods.

Key Results

  • CXR Detection: The ensemble model achieved an accuracy of 90.6%, with a sensitivity (detection rate) of 91.1% and a specificity of 90% on the test set. The Area Under the ROC Curve (AUC) was 0.96.
  • LUS Severity Grading: Their proposed system achieved an F1 score of 68.7% on the ICLUS dataset, surpassing the performance of a more complex Spatial Transformer Network-based model from prior work (which achieved 65.1%).
  • Image Retrieval: KNN applied to image embeddings achieved 87% accuracy and 83.2% sensitivity in retrieving images with the same ground truth label.

Limitations and Future Work

  • CXR Data Limitations: While the CXR dataset is large, the authors acknowledge that it was collected from a limited number of hospitals in Israel. Future work could involve expanding the dataset to include images from more diverse sources and equipment to further enhance generalizability.
  • LUS Data Characteristics: The LUS severity grading work relies on the ICLUS dataset, which has a specific distribution of probe types. Further validation on datasets with different distributions or from different clinical settings would be beneficial.
  • Generalizability: While the models show promising results, their performance on unseen data from different populations or imaging protocols needs further investigation.
  • Clinical Integration: The paper lays the groundwork for POC diagnostics. Future work could focus on integrating these models into real-world clinical workflows and evaluating their impact on patient care and decision-making.

Summary updated

Abstract

Early detection of COVID-19 is key in containing the pandemic. Disease detection and evaluation based on imaging is fast and cheap and therefore plays an important role in COVID-19 handling. COVID-19 is easier to detect in chest CT, however, it is expensive, non-portable, and difficult to dis-infect, making it unfit as a point-of-care (POC) modality. On the other hand, chest X-ray (CXR) and lung ultrasound (LUS) are widely used, yet, COVID-19 findings in these modalities are not always very clear. Here we train deep neural networks to significantly enhance the capability to detect, grade and monitor COVID-19 patients using CXRs and LUS. Collaborating with several hospitals in Israel we collect a large dataset of CXRs and use this dataset to train a neural network obtaining above 90% detection rate for COVID-19. In addition, in collaboration with ULTRa (Ultrasound Laboratory Trento, Italy) and hospitals in Italy we obtained POC ultrasound data with annotations of the severity of disease and trained a deep network for automatic severity grading.
BibTeX
@inproceedings{yaron2021point,
  author = {Daniel Yaron and Daphna Keidar and Elisha Goldstein and Yair Shachar and Ayelet Blass and Oz Frank and Nir Schipper and Nogah Shabshin and Ahuva Grubstein and Dror Suhami and Naama R. Bogot and Chedva S. Weiss and Eyal Sela and Amiel A. Dror and Mordehay Vaturi and Federico Mento and Elena Torri and Riccardo Inchingolo and Andrea Smargiassi and Gino Soldati and Tiziano Perrone and Libertario Demi and Meirav Galun and Shai Bagon and Yishai M. Elyada and Yonina C. Eldar},
  title = {{Point of Care Image Analysis for COVID-19}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2021},
  url = {https://openreview.net/forum?id=2nhG5cr8fO0},
}
Figure