Integrating Domain Knowledge Into Deep Networks for Lung Ultrasound With Applications to COVID-19

Oz Frank, Nir Schipper, Mordehay Vaturi, Gino Soldati, Andrea Smargiassi, Riccardo Inchingolo, Elena Torri, Tiziano Perrone, Federico Mento, Libertario Demi, Meirav Galun, Yonina C. Eldar, Shai Bagon

Abstract

Lung ultrasound (LUS) is a cheap, safe, and non-invasive imaging modality that can be performed at the patient's bedside. However, LUS is not yet widely adopted due to the lack of trained personnel required for interpreting acquired frames. In this work, we propose a framework for training deep artificial neural networks for interpreting LUS, promoting broader clinical use. Our framework integrates domain knowledge into deep neural networks by inputting anatomical features and LUS artifacts as additional channels—specifically, pleural and vertical artifact masks—alongside raw LUS frames. By explicitly supplying this domain knowledge, standard off-the-shelf neural networks can be rapidly and efficiently fine-tuned for tasks like frame classification or semantic segmentation. We evaluated our framework on COVID-19 severity assessment using the ICLUS dataset, demonstrating that our models outperform more complex, custom-designed architectures.

The Challenge of Lung Ultrasound Interpretation

Lung Ultrasound (LUS) is a powerful diagnostic tool, offering portability and safety compared to CT scans. Despite these advantages, interpreting LUS is difficult because it relies on identifying "noisy" sonographic artifacts rather than direct anatomical imaging. This creates a high barrier to entry for medical staff.

While Deep Learning (DL) offers a path toward automation, standard AI models face two major hurdles in this domain:

  1. Data Scarcity: High-quality, annotated medical images are far rarer than the "natural" images used to train general-purpose AI.
  2. Artifact-Driven Logic: In LUS, the spatial context is critical. For instance, a bright horizontal line is interpreted differently depending on whether it is above or below the pleural line. Standard Convolutional Neural Networks (CNNs) often struggle to learn these physics-based spatial relationships from raw pixels alone.

Methodology: Encoding Domain Knowledge

Instead of designing a complex, task-specific neural architecture, we propose a data-centric approach. We augment the raw LUS frame with two additional "domain knowledge" channels, creating a 3-channel input similar to a standard RGB image:

  • Vertical Artifact Mask: We use a preprocessing step to detect vertical "B-lines" (indicators of lung congestion). For convex probes, the image is first "rectified" to Cartesian coordinates so artifacts become strictly vertical, followed by linear fitting to create a binary mask.
  • Pleural Line Signed Distance Mask: The pleural line—the boundary of the lung—is detected using a Radon transform. We then create a mask where each pixel’s value represents its distance from that line. This allows the network to trivially distinguish between exterior tissue and the lung cavity.
Example of a lung ultrasound frame (score 0) showing the pleural line (green arrow) and horizontal A-lines (blue arrow).

By concatenating these masks with the raw frame, standard models like ResNet-18 or MobileNetV2 can be fine-tuned to "attend" to clinically relevant regions immediately, rather than trying to learn ultrasound physics from scratch.

Dataset and Evaluation

Our study utilizes the ICLUS dataset, which includes 277 LUS videos from 35 patients across five medical centers. This dataset contains nearly 60,000 frames captured using both convex and linear probes. Frames are annotated with a 4-level COVID-19 severity score (0 to 3) and pixel-level annotations for semantic segmentation.

Lung ultrasound frames illustrating the COVID-19 severity scoring system from 0 to 3, highlighting various pathological artifacts.

Key Results

The framework was tested on COVID-19 severity classification and semantic segmentation, yielding significant improvements over baseline models:

  • Classification Accuracy: A standard ResNet-18 using our augmented 3-channel input achieved an F1-score of 68.8%, outperforming the baseline ResNet-18 (62.2%) and even beating more complex, custom-designed LUS architectures.
  • Segmentation Precision: For pixel-level prediction, our approach reached a Category Dice score of 0.70, compared to 0.62 for models trained on raw frames alone.
  • Data Efficiency: Models using our framework with only 50% of the training data performed as well as baseline models using 100% of the data. This is a critical advantage in medical AI, where labeled data is expensive and slow to acquire.

Conclusion

By explicitly providing the "where" and "what" of ultrasound artifacts via input channels, we can leverage simple, off-the-shelf neural networks to achieve state-of-the-art results. This approach simplifies the deployment pipeline and makes AI-assisted lung diagnostics more accessible. Future work will focus on incorporating additional artifacts, such as horizontal A-lines, and testing the framework on other pulmonary conditions like pneumonia or heart failure.

Summary updated

Abstract

Lung ultrasound (LUS) is a cheap, safe and non-invasive imaging modality that can be performed at patient bed-side. However, to date LUS is not widely adopted due to lack of trained personnel required for interpreting the acquired LUS frames. In this work we propose a framework for training deep artificial neural networks for interpreting LUS, which may promote broader use of LUS. When using LUS to evaluate a patient’s condition, both anatomical phenomena (e.g., the pleural line, presence of consolidations), as well as sonographic artifacts (such as A- and B-lines) are of importance. In our framework, we integrate domain knowledge into deep neural networks by inputting anatomical features and LUS artifacts in the form of additional channels containing pleural and vertical artifacts masks along with the raw LUS frames. By explicitly supplying this domain knowledge, standard off-the-shelf neural networks can be rapidly and efficiently finetuned to accomplish various tasks on LUS data, such as frame classification or semantic segmentation. Our framework allows for a unified treatment of LUS frames captured by either convex or linear probes. We evaluated our proposed framework on the task of COVID-19 severity assessment using the ICLUS dataset. In particular, we finetuned simple image classification models to predict per-frame COVID-19 severity score. We also trained a semantic segmentation model to predict per-pixel COVID-19 severity annotations. Using the combined raw LUS frames and the detected lines for both tasks, our off-the-shelf models performed better than complicated models specifically designed for these tasks, exemplifying the efficacy of our framework.
BibTeX
@article{frank2022integrating,
  author = {Oz Frank and Nir Schipper and Mordehay Vaturi and Gino Soldati and Andrea Smargiassi and Riccardo Inchingolo and Elena Torri and Tiziano Perrone and Federico Mento and Libertario Demi and Meirav Galun and Yonina C. Eldar and Shai Bagon},
  title = {{Integrating Domain Knowledge Into Deep Networks for Lung Ultrasound With Applications to COVID-19}},
  journal = {IEEE Trans. Medical Imaging},
  year = {2022},
  url = {https://openreview.net/forum?id=ukmZRZdI0p},
}
Figure