Cross-Site Severity Assessment of COVID-19 From CT Images via Domain Adaptation

Gengxin Xu, Chen Liu, Jun Liu, Zhongxiang Ding, Feng Shi, Man Guo, Wei Zhao, Xiaoming Li, Ying Wei, Yaozong Gao, Chuan-Xian Ren, Dinggang Shen

TL;DR

Researchers have developed FLARE, a novel domain adaptation framework that accurately assesses COVID-19 severity from CT scans even when data comes from different hospitals with different equipment. By combining a smart sampling strategy with a multi-pronged representation learning approach, FLARE overcomes the dual hurdles of severe class imbalance and "domain shift" between clinical sites.

What Problem Does This Paper Address?

Assessing the severity of a COVID-19 infection (mild vs. severe) is critical for triaging patients and planning ICU resources. While deep learning models can help, they often struggle when deployed in the real world. Two major roadblocks stand in the way:

  1. Domain Shift: A model trained on CT scans from "Hospital A" (using a Siemens scanner) might fail at "Hospital B" (using a GE scanner) because of subtle differences in image contrast, noise, and protocols.
  2. Class Imbalance: In most clinical settings, mild cases far outnumber severe ones. Standard models tend to become "lazy," over-predicting the majority class (mild) and missing the critical severe cases.

Existing methods usually tackle these problems in isolation. This paper introduces a unified approach to handle both simultaneously.

Methods

The authors propose FLARE (Fair cross-domain adaptation with LAtent REpresentations). The framework consists of two main pillars:

  • Stochastic Class-balanced Boosting Sampling (SCBS): Instead of simple oversampling, SCBS dynamically adjusts which samples the model sees during training. It prioritizes not just the minority class (severe cases) but also "hard" samples that the model is currently struggling to predict correctly.
  • Advanced Representation Learning: The model projects complex CT features into a "latent space" designed to have three specific properties:
  • Domain-transferability: It uses a domain translator and a prototype triplet loss to align data from different hospitals around shared "class centers" (prototypes).
  • Discriminative Power: It employs a Conditional Maximum Mean Discrepancy (CMMD) loss to ensure that the features for "mild" and "severe" stay clearly separated.
  • Completeness: A multi-view reconstruction loss forces the latent representation to retain enough information to "rebuild" the original input features, ensuring no vital clinical data is lost during the translation.

Datasets

The study utilized a robust mix of private and public data:

  • Three Private Sites: 5,358 CT images collected from three major Chinese hospitals (Second Xiangya Hospital, Hangzhou First People’s Hospital, and Southwest Hospital).
  • One Public Site: The MosMedData dataset from Moscow, Russia, which provided an additional 1,110 CT scans for external validation.
  • Features: Rather than using raw pixels, the authors extracted 237-dimensional "multi-view" features, including texture, shape, and volume metrics, to incorporate expert radiological knowledge.

Key Insights and Ideas

  • The "Hard Sample" Focus: The authors realized that class imbalance isn't just about numbers; it's about difficulty. By boosting the importance of poorly predicted samples over time, the model learns the subtle nuances that distinguish a "borderline" mild case from a severe one.
  • Prototypes as Anchors: Instead of trying to force all source data to look like target data (which can lead to "negative transfer"), FLARE estimates the "average" mild and severe cases (prototypes) in the target hospital and pulls the source data toward those anchors.
  • Handcrafted vs. Deep Features: While many modern papers use end-to-end deep learning, this paper argues that handcrafted features (like lesion volume and surface area) are highly effective for COVID-19 because they directly map to clinical diagnostic criteria.

Key Results

FLARE consistently outperformed state-of-the-art domain adaptation methods (like DAN, DANN, and CDAN) across multiple transfer tasks:

  • Superiority in G-mean: On the most challenging tasks, FLARE achieved G-mean scores (a balance of sensitivity and specificity) of over 86%, significantly higher than standard supervised models which often hovered in the 70s.
  • Robustness to Small Data: Even when the "target" hospital had very few labeled images, FLARE was able to "borrow" knowledge from other sites effectively to maintain high accuracy.
  • Multi-Source Success: An extended version, M-FLARE, showed that the model gets even better as you add more source hospitals, effectively aggregating knowledge while ignoring the "noise" of different scanner types.

Limitations and Future Work

The authors acknowledge a few areas for growth:

  • Handcrafted Feature Dependency: While effective, extracting these features requires a pre-processing step (segmentation) that could be integrated into a fully end-to-end deep learning pipeline in the future.
  • Static vs. Dynamic: The current model looks at a single point in time. Future work could incorporate the progression of the disease by analyzing sequences of CT scans taken over several days.
  • Synthetic Data: The authors suggest that instead of just re-sampling real data, future versions could use Generative Adversarial Networks (GANs) to create synthetic "severe" cases to further balance the training sets.

Summary updated

Abstract

Early and accurate severity assessment of Coronavirus disease 2019 (COVID-19) based on computed tomography (CT) images offers a great help to the estimation of intensive care unit event and the clinical decision of treatment planning. To augment the labeled data and improve the generalization ability of the classification model, it is necessary to aggregate data from multiple sites. This task faces several challenges including class imbalance between mild and severe infections, domain distribution discrepancy between sites, and presence of heterogeneous features. In this paper, we propose a novel domain adaptation (DA) method with two components to address these problems. The first component is a stochastic class-balanced boosting sampling strategy that overcomes the imbalanced learning problem and improves the classification performance on poorly-predicted classes. The second component is a representation learning that guarantees three properties: 1) domain-transferability by prototype triplet loss, 2) discriminant by conditional maximum mean discrepancy loss, and 3) completeness by multi-view reconstruction loss. Particularly, we propose a domain translator and align the heterogeneous data to the estimated class prototypes (i.e., class centers) in a hyper-sphere manifold. Experiments on cross-site severity assessment of COVID-19 from CT images show that the proposed method can effectively tackle the imbalanced learning problem and outperform recent DA approaches.
BibTeX
@article{xu2022cross,
  author = {Gengxin Xu and Chen Liu and Jun Liu and Zhongxiang Ding and Feng Shi and Man Guo and Wei Zhao and Xiaoming Li and Ying Wei and Yaozong Gao and Chuan-Xian Ren and Dinggang Shen},
  title = {{Cross-Site Severity Assessment of COVID-19 From CT Images via Domain Adaptation}},
  journal = {IEEE Trans. Medical Imaging},
  year = {2022},
  url = {https://openreview.net/forum?id=k584OjUAO7},
}
Figure