Federated semi-supervised learning for COVID region segmentation in chest CT using multi-national data from China, Italy, Japan

Dong Yang, Ziyue Xu, Wenqi Li, Andriy Myronenko, Holger R. Roth, Stephanie A. Harmon, Sheng Xu, Baris Turkbey, Evrim Turkbey, Xiaosong Wang, Wentao Zhu, Gianpaolo Carrafiello, Francesca Patella, Maurizio Cariati, Hirofumi Obinata, Hitoshi Mori, Kaku Tamura, Peng An, Bradford J. Wood, Daguang Xu

TL;DR

This paper introduces a Federated Semi-Supervised Learning (FSSL) framework designed to segment COVID-19 lesions in chest CT scans using data from China, Italy, and Japan. By combining federated learning—which keeps sensitive patient data private—with semi-supervised techniques that utilize unlabeled images, the researchers created a model that generalizes across different countries and clinical settings with minimal human annotation.

What Problem Does This Paper Address?

Deep learning models for medical imaging usually face two massive hurdles: privacy and data scarcity.

  1. Privacy: Hospitals cannot easily share raw patient data due to strict regulations (like HIPAA or GDPR), leading to "data silos."
  2. Data Scarcity & Heterogeneity: Manually labeling thousands of 3D CT scans is exhausting for radiologists. Furthermore, a model trained on data from one country often fails when applied to another because of "domain shift"—differences in CT equipment, scanning protocols, and patient demographics.

The authors tackle these by asking: Can we build a robust, global COVID-19 segmentation model without moving data out of hospitals and without requiring every single image to be labeled?

Methods

The authors propose a novel FSSL system that integrates several moving parts:

  • Federated Learning (FL): Using a server-client architecture, the "global" model is sent to local hospitals (clients). Clients train on their local data and send only the model weights/gradients back to the server for aggregation (Weighted Federated Averaging). No raw images ever leave the hospital.
  • Consistency-Based Semi-Supervision: To use unlabeled data, the authors employ a "teacher-student" logic. The model generates a "pseudo-label" for an unlabeled image. The image is then perturbed (augmented), and the model is trained to ensure its prediction on the noisy image remains consistent with the original pseudo-label.
  • Global Semi-Supervision: Unlike traditional semi-supervision where one client has both labeled and unlabeled data, this framework allows for "unsupervised clients"—hospitals that contribute only unlabeled data to the global effort.
  • Baseline Architecture: A 3D U-shape Fully Convolutional Network (FCN), partially initialized with pre-trained ResNet weights.

Datasets

The study utilized a massive multi-national database of 1,704 total scans:

  • COVID-19 Cohorts:
    • Image_1 (China): 736 scans (671 annotated).
  • Image_2 (Japan): 496 scans (88 annotated).
  • Image_3 (Italy): 472 scans (186 annotated).
  • Control Populations: To reduce false positives, they included scans of "normal" lungs, other pneumonias (bacteria/fungi), and lung nodules (from the public LIDC dataset).
  • Annotation: Expert radiologists manually delineated 945 scans to provide the ground truth for training and validation.

Key Insights and Ideas

  • The Power of the Unlabeled: One of the most striking findings is that adding unlabeled data from a different country (e.g., Italy) helps a model trained on Chinese data generalize much better than simply adding more labeled data from the same Chinese site.
  • "Global" vs. "Local" Semi-Supervision: The authors distinguish between these two. In a global setting, the diversity of unlabeled data across different institutions acts as a natural regularizer, helping the model find a "low-error space" that works for everyone.
  • Sensitivity to Learning Rates: Unsupervised clients are delicate. The authors found that the learning rate for unlabeled data must be significantly lower (e.g., $5 \times 10^{-6}$) than for labeled data to prevent the model from diverging or overfitting to noisy pseudo-labels.

Key Results

  • Performance Gap: The FSSL approach achieved results nearly identical to "Fully Supervised" Federated Learning, with only a ~1% drop in Dice score, despite using significantly fewer annotations.
  • Generalizability: Models trained via FSSL on two countries performed significantly better on a third, "unseen" country compared to models trained on a single site. For example, a model trained on Chinese data saw a 10% drop in accuracy when tested on Japanese data; FSSL mitigated this performance decay.
  • Ablation Success: Hard Dice loss was found to be the most effective loss function for consistency training, and frequent model synchronization (every 5-10 epochs) led to smoother training trajectories.

Limitations and Future Work

  • Weight Divergence: The authors acknowledge that gradients from different clients can diverge if the data is too "non-IID" (not identically and independently distributed), which remains a challenge in federated learning.
  • Hyperparameter Tuning: The system is sensitive to the "aggregation weights" (how much the server trusts each hospital's update). Currently, this requires empirical tuning.
  • Future Directions: The authors suggest exploring Federated Unsupervised Learning for representation learning, where a model could learn the "features" of human lungs across the globe without any labels at all before being fine-tuned for specific diseases.

Summary updated

Abstract

Highlights • First work to investigate solutions for collaborative medical AI model for COVID-19. • A novel federated semi-supervised learning technique is proposed to train a network. • Method fully utilizes available data from different sources, w/ or w/o annotations. • Multinational dataset of 1704 scans from 3 countries (China, Italy, and Japan). • Studies fully evaluate and understand different configurations of the method. Abstract The recent outbreak of Coronavirus Disease 2019 (COVID-19) has led to urgent needs for reliable diagnosis and management of SARS-CoV-2 infection. The current guideline is using RT-PCR for testing. As a complimentary tool with diagnostic imaging, chest Computed Tomography (CT) has been shown to be able to reveal visual patterns characteristic for COVID-19, which has definite value at several stages during the disease course. To facilitate CT analysis, recent efforts have focused on computer-aided characterization and diagnosis with chest CT scan, which has shown promising results. However, domain shift of data across clinical data centers poses a serious challenge when deploying learning-based models. A common way to alleviate this issue is to fine-tune the model locally with the target domains local data and annotations. Unfortunately, the availability and quality of local annotations usually varies due to heterogeneity in equipment and distribution of medical resources across the globe. This impact may be pronounced in the detection of COVID-19, since the relevant patterns vary in size, shape, and texture. In this work, we attempt to find a solution for this challenge via federated and semi-supervised learning. A multi-national database consisting of 1704 scans from three countries is adopted to study the performance gap, when training a model with one dataset and applying it to another. Expert radiologists manually delineated 945 scans for COVID-19 findings. In handling the variability in both the data and annotations, a novel federated semi-supervised learning technique is proposed to fully utilize all available data (with or without annotations). Federated learning avoids the need for sensitive data-sharing, which makes it favorable for institutions and nations with strict regulatory policy on data privacy. Moreover, semi-supervision potentially reduces the annotation burden under a distributed setting. The proposed framework is shown to be effective compared to fully supervised scenarios with conventional data sharing instead of model weight sharing.
BibTeX
@article{yang2021federated,
  author = {Dong Yang and Ziyue Xu and Wenqi Li and Andriy Myronenko and Holger R. Roth and Stephanie A. Harmon and Sheng Xu and Baris Turkbey and Evrim Turkbey and Xiaosong Wang and Wentao Zhu and Gianpaolo Carrafiello and Francesca Patella and Maurizio Cariati and Hirofumi Obinata and Hitoshi Mori and Kaku Tamura and Peng An and Bradford J. Wood and Daguang Xu},
  title = {{Federated semi-supervised learning for COVID region segmentation in chest CT using multi-national data from China, Italy, Japan}},
  journal = {Medical Image Analysis},
  year = {2021},
  url = {https://openreview.net/forum?id=j-ZiSQcZdf6},
}
Figure