FedFTN: Personalized Federated Learning with Deep Feature Transformation Network for Multi-institutional Low-count PET Denoising

Bo Zhou, Huidong Xie, Qiong Liu, Xiongchao Chen, Xueqi Guo, Zhicheng Feng, S. Kevin Zhou, Biao Li, Axel Rominger, Kuangyu Shi, James S. Duncan, Chi LiuMedical Image Analysis 2023 · OpenReview

TL;DR

This paper introduces FedFTN, a novel federated learning framework for denoising low-count PET images. By employing personalized "Feature Transformation Networks" (FTNs) at each institution, FedFTN effectively addresses domain shift issues and enhances image quality without compromising patient privacy, outperforming existing federated learning methods.

What Problem Does This Paper Address?

Positron Emission Tomography (PET) is a crucial medical imaging technique, but low-count PET scans, used to minimize radiation exposure and scan time, often result in images with low signal-to-noise ratios (SNR). This noise can hinder accurate diagnosis and downstream analyses. While deep learning has shown promise in denoising these images, training robust models typically requires large, centralized datasets. However, sharing patient data across institutions is fraught with privacy and security concerns. Furthermore, data distributions can vary significantly between institutions due to differences in PET scanners, protocols, and reconstruction methods, necessitating personalized models. Existing federated learning (FL) approaches allow collaborative training without data sharing but often struggle with these domain shifts, especially in complex tasks like PET denoising. This paper aims to bridge this gap by proposing a personalized FL strategy specifically for multi-institutional low-count PET denoising.

Methods

The core innovation of FedFTN lies in its Feature Transformation Network (FTN). This network is designed to personalize the denoising process at each institution. The overall architecture consists of a globally shared denoising network (a U-Net in this case) and local FTNs. During federated training, the FTNs are kept local to each institution, while only the parameters of the shared denoising network are communicated and aggregated by a central server.

Here's how it works:

  • FTN's Role: The FTN takes intermediate features from the denoising network and modulates them. It also incorporates the specific low-count level of the PET data as an input, allowing for dose-level-aware personalization. This modulation helps adapt the denoising network to the unique data distribution of each institution and the specific noise characteristics of different low-count levels.
  • Federated Learning Pipeline: The process involves local training at each institution, followed by parameter aggregation at the central server. To stabilize training, a Global Weight Constraint (GWC) loss is introduced, which encourages local denoising network weights to stay close to the aggregated global weights.
  • Personalization: The FTNs are trained locally, ensuring that each institution develops a personalized denoising model tailored to its specific data and low-count protocols.

Datasets

The study utilizes a large-scale, multi-institutional dataset comprising low-count PET imaging data from three medical centers located in:

  • Yale New Haven Hospital, USA: 200 subjects with 18F-FDG tracer.
  • University of Bern, Switzerland: 209 subjects with 18F-FDG tracer.
  • Ruijin Hospital, Shanghai, China: 204 subjects with 18F-FDG tracer.

For each institution, low-count PET data was generated by down-sampling the original PET list-mode data to create three distinct low-count levels (e.g., 5%, 10%, 20% or 2%, 5%, 10%). Full-count PET images were used as the ground truth for evaluation. This dataset represents a significant contribution as it is a real-world, multi-institutional benchmark for low-count PET denoising.

Key Insights and Ideas

  • Personalization is Key for Domain Shift: The paper strongly demonstrates that simply aggregating models in a federated manner isn't enough when data distributions vary significantly. Personalized components, like the FTN, are crucial for achieving high performance across diverse institutions.
  • FTN as a Flexible Personalization Module: The FTN's ability to adapt to both institutional differences and varying low-count levels within an institution is a significant advantage. It acts as a lightweight, local module that allows for sophisticated personalization without sharing raw data.
  • Low-Count Level as Input: Incorporating the low-count level directly into the personalization process is an intuitive and effective way to handle the varying noise characteristics of different scan settings.
  • Federated Learning for Complex 3D Medical Imaging: This work extends the application of federated learning to the challenging domain of 3D medical image reconstruction/denoising, which is often more complex than 2D tasks like image classification.
  • GWC Loss for Stability: The Global Weight Constraint loss is a valuable addition for stabilizing federated training, particularly in scenarios with significant domain shifts, leading to more robust and consistent performance.

Key Results

  • Superior Performance: FedFTN consistently outperformed previous baseline FL reconstruction methods (FedAvg, FedBN, FedProx, FedSP, FedHyper) and locally trained models across all low-count levels at all three institutions.
  • Quantitative Improvements: FedFTN achieved statistically significant improvements in image quality metrics like PSNR and NMSE compared to baselines. For instance, at Institution #1, FedFTN improved PSNR from 26.02 (local unified model) to 27.24 for the 5% low-count PET.
  • Qualitative Improvements: Visual comparisons showed that FedFTN produced denoised images with sharper details, better recovery of general structures, and reduced artifacts (e.g., false positives in cardiac defects or blurring in bone marrow) compared to other methods.
  • Site Adaptation Boost: Further fine-tuning FedFTN models with local data (Site Adaptation) provided incremental but noticeable improvements in image quality metrics.
  • GWC Loss Benefits: Ablation studies showed that the GWC loss consistently improved image quality metrics and led to faster, more stable training loss convergence.

Limitations and Future Work

  • Limited to 18F-FDG Tracer: The study exclusively used 18F-FDG tracer data. The authors suggest future work could explore adapting FedFTN to other PET tracers and incorporating tracer type as an input to the FTN for even greater personalization.
  • Three Institutions: While multi-institutional, the study was limited to three sites. Expanding to more institutions would further validate the generalizability of the approach.
  • Focus on Image Quality Metrics: The evaluation primarily relied on image quality metrics (PSNR, NMSE). Future work should investigate how these denoising improvements translate to downstream clinical tasks like disease quantification, staging, and therapy response assessment, and include human expert evaluations.
  • U-Net Backbone: The experiments used a U-Net as the denoising network. The authors propose adapting FedFTN to more advanced restoration network structures (e.g., cascade, transformer-based) to potentially achieve even better performance.

Summary updated

Abstract

Low-count PET is an efficient way to reduce radiation exposure and acquisition time, but the reconstructed images often suffer from low signal-to-noise ratio (SNR), thus affecting diagnosis and other downstream tasks. Recent advances in deep learning have shown great potential in improving low-count PET image quality, but acquiring a large, centralized, and diverse dataset from multiple institutions for training a robust model is difficult due to privacy and security concerns of patient data. Moreover, low-count PET data at different institutions may have different data distribution, thus requiring personalized models. While previous federated learning (FL) algorithms enable multi-institution collaborative training without the need of aggregating local data, addressing the large domain shift in the application of multi-institutional low-count PET denoising remains a challenge and is still highly under-explored. In this work, we propose FedFTN, a personalized federated learning strategy that addresses these challenges. FedFTN uses a local deep feature transformation network (FTN) to modulate the feature outputs of a globally shared denoising network, enabling personalized low-count PET denoising for each institution. During the federated learning process, only the denoising network's weights are communicated and aggregated, while the FTN remains at the local institutions for feature transformation. We evaluated our method using a large-scale dataset of multi-institutional low-count PET imaging data from three medical centers located across three continents, and showed that FedFTN provides high-quality low-count PET images, outperforming previous baseline FL reconstruction methods across all low-count levels at all three institutions.
BibTeX
@article{zhou2023fedftn,
  author = {Bo Zhou and Huidong Xie and Qiong Liu and Xiongchao Chen and Xueqi Guo and Zhicheng Feng and S. Kevin Zhou and Biao Li and Axel Rominger and Kuangyu Shi and James S. Duncan and Chi Liu},
  title = {{FedFTN: Personalized Federated Learning with Deep Feature Transformation Network for Multi-institutional Low-count PET Denoising}},
  journal = {Medical Image Analysis},
  year = {2023},
  url = {https://openreview.net/forum?id=twzA11CkST},
}
Figure