Self-Supervised Pretraining Improves Performance and Inference Efficiency in Multiple Lung Ultrasound Interpretation Tasks

Blake VanBerlo, Brian Li, Jesse Hoey, Alexander Wong

This paper, authored by researchers at the University of Waterloo, explores how self-supervised learning can make lung ultrasound (LUS) AI more accurate, data-efficient, and fast enough for real-time clinical use.

TL;DR

By using self-supervised pretraining (SSL), the authors developed a feature extractor that improves lung ultrasound classification performance, especially when labeled data is scarce. Furthermore, by sharing a single "frozen" feature extractor across multiple diagnostic tasks, they reduced inference time by 49% without significantly sacrificing accuracy.

What Problem Does This Paper Address?

Lung ultrasound (LUS) is a critical tool in emergency and intensive care, but interpreting the images requires significant expertise. While deep learning can help, two major hurdles remain:

  1. The Label Scarcity Gap: Labeling medical images is expensive and requires expert clinicians. Standard supervised learning requires massive amounts of labeled data to perform well.
  2. Inference Efficiency: Clinical LUS interpretation is hierarchical (e.g., first determine the "view," then look for specific artifacts like A-lines or pleural effusions). Running separate, full-sized neural networks for every step in this decision tree is computationally expensive and slow for point-of-care devices.

The authors investigate whether Self-Supervised Learning (SSL)—specifically joint-embedding methods—can create a "universal" feature extractor for LUS that solves both problems.

Methods

The researchers employed a MobileNetV3 architecture, chosen for its balance of performance and efficiency on mobile hardware. Their approach involved two main phases:

  1. Pretraining: They used three popular SSL methods—SimCLR, Barlow Twins, and VICReg—to train the model on a large pool of unlabeled LUS images. These methods teach the model to recognize underlying patterns by comparing different "augmented" views of the same image.
  2. Downstream Tasks: They evaluated the pretrained model on three binary classification tasks:
  3. View: Distinguishing between Parenchymal and Pleural views.
  4. A/B: Identifying normal (A-lines) vs. abnormal (B-lines) lung tissue.
  5. PE: Detecting the presence or absence of Pleural Effusion.

They compared Fine-Tuning (FT) (updating the whole model) against Linear (LC) and Nonlinear Classification (NC) (freezing the backbone and only training a small "head").

Datasets

The study utilized a robust mix of data:

  • Institutional Dataset (Local): A private repository of LUS videos containing over 2.1 million unlabeled images for pretraining and a labeled subset of ~750k images for training/validation.
  • External Dataset: A separate institutional dataset used strictly for testing to evaluate how well the model generalizes to different clinical environments.
  • COVIDxUS: A public benchmark dataset used to test the transferability of the pretrained weights to a completely different data source.

Key Insights and Ideas

  • SSL as a "Force Multiplier" for Small Data: The most dramatic gains occurred when using only 1% of available labels. In this "low-data" regime, SSL-pretrained models crushed standard ImageNet-initialized models, with the View classification AUC increasing by a staggering 0.396.
  • The Power of Shared Representations: The study proves that a single feature extractor can capture enough information to serve multiple different diagnostic tasks. This challenges the traditional approach of training "siloed" models for every specific medical condition.
  • Efficiency via "Frozen" Backbones: By freezing the feature extractor and only training tiny nonlinear "heads" (MLPs) for specific tasks, the researchers created a system that is much faster. Since the heavy lifting (feature extraction) is only done once per image, the marginal cost of adding new diagnostic tasks becomes negligible.

Key Results

  • Performance Boost: On average, SimCLR pretraining improved the Area Under the Curve (AUC) by 0.032 on local tests and 0.061 on external tests compared to models without SSL pretraining.
  • Speedup: Reusing a single feature extractor for multiple tasks reduced inference time by 49% (from 0.116s to 0.059s for 1,000 predictions) compared to running separate models serially.
  • Generalization: The SSL models pretrained on LUS data significantly outperformed models initialized with standard ImageNet weights when tested on the public COVIDxUS dataset, proving that domain-specific pretraining is superior to general-purpose pretraining.

Limitations and Future Work

  • The Pleural Effusion (PE) Challenge: While SSL helped most tasks, the "frozen" nonlinear classifiers struggled to outperform fully fine-tuned models on Pleural Effusion detection. t-SNE visualizations showed that PE features were less "separable" than others, suggesting this specific task might require more specialized features.
  • 2D vs. 3D: The current study treats ultrasound as a series of 2D images. The authors note that future work should exploit the temporal nature of ultrasound videos (3D/Video-based SSL) to potentially capture even richer representations.
  • Augmentation Strategy: The study used standard computer vision augmentations (crops, flips). The authors suggest that developing ultrasound-specific augmentations (simulating acoustic noise or probe artifacts) could further improve SSL performance.

Summary updated

Abstract

In this study, we investigated whether self-supervised pretraining could produce a neural network feature extractor applicable to multiple classification tasks in B-mode lung ultrasound analysis. When fine-tuning on three lung ultrasound tasks, pretrained models resulted in an improvement of the average across-task area under the receiver operating curve (AUC) by 0.032 and 0.061 on local and external test sets respectively. Compact nonlinear classifiers trained on features outputted by a single pretrained model did not improve performance across all tasks; however, they did reduce inference time by 49% compared to serial execution of separate fine-tuned models. When training using 1% of the available labels, pretrained models consistently outperformed fully supervised models, with a maximum observed test AUC increase of 0.396 for the task of view classification. Overall, the results indicate that self-supervised pretraining is useful for producing initial weights for lung ultrasound classifiers.
BibTeX
@article{vanberlo2023self,
  author = {Blake VanBerlo and Brian Li and Jesse Hoey and Alexander Wong},
  title = {{Self-Supervised Pretraining Improves Performance and Inference Efficiency in Multiple Lung Ultrasound Interpretation Tasks}},
  journal = {IEEE Access},
  year = {2023},
  doi = {10.1109/access.2023.3337398},
}

Loading related content...

Figure