Federated Learning for Thyroid Ultrasound Image Analysis to Protect Personal Information: Validation Study in a Real Health Care Environment (Preprint)

Haeyun Lee, Young Jun Chai, Hyunjin Joo, Kyungsu Lee, Jae Youn Hwang, Seok‐Mo Kim, Kwangsoon Kim, Inn‐Chul Nam, June Young Choi, Hyeong Won Yu, Myung‐Chul Lee, Hiroo Masuoka, Akira Miyauchi, Sungwan Kim, Hyoun‐Joong Kong2020 · DOI

TL;DR

This paper explores Federated Learning (FL) for thyroid nodule classification from ultrasound images, demonstrating its potential to achieve comparable performance to traditional deep learning while preserving patient data privacy. By training models across multiple institutions without centralizing sensitive medical data, FL offers a promising solution for real-world healthcare applications.

What Problem Does This Paper Address?

The increasing reliance on deep learning for medical image analysis faces a significant hurdle: data privacy. Regulations like HIPAA and GDPR make it challenging to collect and centralize large, diverse datasets from multiple healthcare institutions, which are crucial for training robust and generalizable models. This data fragmentation leads to models trained on limited, institution-specific data, potentially hindering their performance and applicability across different patient populations and imaging equipment. The paper addresses this by investigating Federated Learning, a decentralized approach that allows models to be trained collaboratively across institutions without ever sharing raw patient data.

Methods

The core of this study lies in the application and evaluation of Federated Learning for thyroid nodule classification. The authors employed a federated learning framework where each participating institution acted as a "serverworker." These serverworkers trained deep learning models locally on their own data. The trained model parameters (not the data itself) were then sent to a central "coordinator" (located at Seoul National University Hospital). The coordinator aggregated these parameters to create an updated global model, which was then sent back to the serverworkers for the next round of training. This iterative process of local training and global aggregation is the essence of federated learning.

For the deep learning models, the researchers utilized five established convolutional neural network architectures: VGG19, ResNet50, ResNext50, SE-ResNet50, and SE-ResNext50. They employed the ADAM optimizer with specific learning rate schedules and mini-batch sizes. To compare FL's performance, they also trained the same five models using a conventional deep learning approach, where all data from the participating institutions were pooled into a single dataset.

Datasets

The study utilized a substantial dataset of 8457 thyroid ultrasound images collected from six different medical institutions. This dataset comprised 5375 malignant and 3082 benign nodules. For internal validation, 20% of this data (1075 malignant, 616 benign) was set aside. For external validation, a separate set of 100 ultrasound images (50 malignant, 50 benign) from a medical institution in Japan was used. The images were cropped to 299x299 pixels and were not augmented.

Key Insights and Ideas

  • Privacy-Preserving Collaboration: The most significant insight is the demonstration that federated learning can effectively train high-performing medical image analysis models without compromising patient privacy. This opens doors for broader collaboration among healthcare providers.
  • Bridging Data Heterogeneity: The study highlights FL's ability to handle diverse data distributions across institutions. Even with significant variations in data volume and class imbalance (e.g., a 47:1 malignant to benign ratio in one institution vs. 1:2 in another), FL showed robust performance.
  • Real-World Validation: Unlike many prior FL studies that used general image datasets or limited medical image data from a single institution, this research validates FL in a real-world healthcare environment with multi-institutional clinical data.
  • Generalizability: The use of an external validation set from a different country (Japan) suggests that FL can contribute to building models with better generalizability across diverse clinical settings and potentially different imaging equipment.

Key Results

  • Comparable Performance: For internal validation, the Area Under the Receiver Operating Characteristic (AUROC) curves for federated learning ranged from 78.88% to 87.56%, which was comparable to conventional deep learning (82.61% to 91.57%).
  • External Validation Success: On the external test set, federated learning achieved AUROC values between 75.20% and 86.72%, again showing comparable performance to conventional deep learning (73.04% to 91.04%).
  • Model Performance: Among the tested models, ResNext50 and SE-ResNext50 generally showed higher AUROC values in both federated and conventional learning scenarios.
  • Training Time: Federated learning training took at least four times longer than conventional deep learning.

Limitations and Future Work

The authors acknowledge several limitations:

  • Generalizability of Results: The study's context (number of institutions, data distribution) might limit the generalizability of the findings. Further investigation with more diverse settings is needed.
  • Image Modality: The study focused on ultrasound images, which are relatively easier to analyze compared to CT, MRI, or histopathology. Future work should explore FL with these more complex modalities.
  • Data Imbalance and Augmentation: While FL handled data imbalance, further research is needed on optimal strategies for data augmentation and addressing extreme imbalances.
  • Training Time: The longer training time for FL is a practical consideration. Exploring ways to optimize this, perhaps through more efficient aggregation or network architectures, is important.

Future work could also explore the implications of unequal data distribution, one-shot learning, and investigate the "black box" nature of deep learning models to understand performance tendencies.

Summary updated

Abstract

<sec> <title>BACKGROUND</title> Federated learning is a decentralized approach to machine learning; it is a training strategy that overcomes medical data privacy regulations and generalizes deep learning algorithms. Federated learning mitigates many systemic privacy risks by sharing only the model and parameters for training, without the need to export existing medical data sets. In this study, we performed ultrasound image analysis using federated learning to predict whether thyroid nodules were benign or malignant. </sec> <sec> <title>OBJECTIVE</title> The goal of this study was to evaluate whether the performance of federated learning was comparable with that of conventional deep learning. </sec> <sec> <title>METHODS</title> A total of 8457 (5375 malignant, 3082 benign) ultrasound images were collected from 6 institutions and used for federated learning and conventional deep learning. Five deep learning networks (VGG19, ResNet50, ResNext50, SE-ResNet50, and SE-ResNext50) were used. Using stratified random sampling, we selected 20% (1075 malignant, 616 benign) of the total images for internal validation. For external validation, we used 100 ultrasound images (50 malignant, 50 benign) from another institution. </sec> <sec> <title>RESULTS</title> For internal validation, the area under the receiver operating characteristic (AUROC) curve for federated learning was between 78.88% and 87.56%, and the AUROC for conventional deep learning was between 82.61% and 91.57%. For external validation, the AUROC for federated learning was between 75.20% and 86.72%, and the AUROC curve for conventional deep learning was between 73.04% and 91.04%. </sec> <sec> <title>CONCLUSIONS</title> We demonstrated that the performance of federated learning using decentralized data was comparable to that of conventional deep learning using pooled data. Federated learning might be potentially useful for analyzing medical images while protecting patients’ personal information. </sec>
BibTeX
@misc{lee2020federated,
  author = {Haeyun Lee and Young Jun Chai and Hyunjin Joo and Kyungsu Lee and Jae Youn Hwang and Seok‐Mo Kim and Kwangsoon Kim and Inn‐Chul Nam and June Young Choi and Hyeong Won Yu and Myung‐Chul Lee and Hiroo Masuoka and Akira Miyauchi and Kyu Eun Lee and Sungwan Kim and Hyoun‐Joong Kong},
  title = {{Federated Learning for Thyroid Ultrasound Image Analysis to Protect Personal Information: Validation Study in a Real Health Care Environment (Preprint)}},
  year = {2020},
  doi = {10.2196/preprints.25869},
}
Figure