Split-U-Net: Preventing Data Leakage in Split Learning for Collaborative Multi-modal Brain Tumor Segmentation

Holger R. Roth, Ali Hatamizadeh, Ziyue Xu, Can Zhao, Wenqi Li, Andriy Myronenko, Daguang Xu

Introduction

In the medical domain, the development of robust deep learning models is often hindered by the fragmentation of data across various institutions. While large-scale datasets are essential for training accurate models, strict privacy regulations such as HIPAA and GDPR prevent the sharing of raw patient information. While Horizontal Federated Learning (HFL) addresses scenarios where sites have similar data types for different patients, many clinical collaborations involve Vertical Federated Learning (VFL). In VFL, different institutions hold different features or modalities—such as T1-weighted, T2, or FLAIR MRI scans—for the same patient cohort.

Split Learning (SL) has emerged as a promising framework for VFL, allowing institutions to train a joint model by sharing intermediate neural network activations rather than raw data. However, a critical security vulnerability remains: the potential for a malicious participant to perform an "inversion attack," reconstructing private source images from intercepted activations. This paper introduces Split-U-Net, a framework designed to enable collaborative multi-modal brain tumor segmentation while quantifying and neutralizing these privacy risks.

Methodology

The Split-U-Net Architecture

The authors adapt the standard U-Net architecture for a multi-party, split-learning environment. The architecture is partitioned into two main components:

  1. Client-Side Encoders: Each participating institution (client) maintains a local encoder that processes its specific MRI modality. These encoders generate intermediate feature maps.
  2. Server-Side Decoder: The feature maps from all clients are transmitted to a central server, which holds the ground-truth segmentation labels. The server concatenates these features and passes them through the U-Net decoder to produce the final segmentation and calculate the loss.

Quantifying Privacy Risk: The Inversion Attack

To evaluate the vulnerability of the shared signals, the researchers developed a white-box inversion attack. In this scenario, an attacker attempts to reconstruct the original input image by optimizing a random noise image until its activations through the client's encoder match the intercepted activations. This serves as a worst-case benchmark for data leakage.

Defense Mechanisms

Three primary strategies were evaluated to mitigate the success of inversion attacks:

  • Architectural Depth: Sharing activations only from deeper, lower-resolution layers of the encoder.
  • Dropout: Applying high rates of dropout during training to introduce stochastic noise into the shared signal.
  • Differential Privacy (DP): Adding calibrated Gaussian noise to the activations before transmission to the server.

Experimental Setup

The framework was evaluated using the Medical Segmentation Decathlon (MSD) brain tumor dataset (Task 1), which contains 484 multi-modal MRI scans (T1, T1ce, T2, and FLAIR). For computational efficiency, the authors formulated a 2D semantic segmentation task by extracting axial slices through the center of the tumors. The data was partitioned into 70% training, 10% validation, and 20% testing sets.

Key Insights and Results

Performance Parity

One of the most significant findings is that Split-U-Net maintains high utility despite the decentralized setup. The model achieved a Dice score of 0.693, nearly matching the performance of a centralized U-Net (0.698) trained on the entire dataset in a single location. This demonstrates that splitting the architecture does not inherently degrade segmentation accuracy.

The "Resolution" of Privacy

The study revealed a clear trade-off between network depth and privacy. Reconstructions from early layers ($x_0$) were virtually indistinguishable from the original images, with a Structural Similarity Index (SSIM) near 1.0. However, as the split point moved deeper into the network, the semantic information increased while spatial detail decreased. By the bottleneck layer, inversion attacks yielded only blurry, non-identifiable blobs.

Effectiveness of Defenses

Differential Privacy (DP) emerged as a particularly effective defense. At a noise level of $\sigma=2.0$, the success of inversion attacks dropped significantly (SSIM falling from ~0.9 to ~0.2). Interestingly, DP also acted as a form of regularization, slightly improving the model's testing performance. High Dropout rates ($p=0.8$) were similarly effective at "blinding" the attacker, reducing the SSIM of reconstructed images to near-zero.

Discussion and Limitations

While Split-U-Net proves effective, several challenges remain for real-world deployment. The current study assumes a white-box attack model; while this is a rigorous security test, black-box scenarios may be more common in practice. Additionally, the transition from 2D slices to full 3D volumes will increase communication overhead between clients and the server.

Furthermore, the authors noted the risk of "label leakage," where gradients sent back from the server might inadvertently reveal information about the tumor masks. Finally, the method assumes that images across different sites are perfectly co-registered, a pre-processing step that remains non-trivial in multi-center clinical studies. Despite these hurdles, Split-U-Net provides a robust foundation for secure, collaborative medical AI.

Summary updated

BibTeX
@inproceedings{roth2022split,
  author = {Holger R. Roth and Ali Hatamizadeh and Ziyue Xu and Can Zhao and Wenqi Li and Andriy Myronenko and Daguang Xu},
  title = {{Split-U-Net: Preventing Data Leakage in Split Learning for Collaborative Multi-modal Brain Tumor Segmentation}},
  booktitle = {DeCaF/FAIR @ MICCAI},
  year = {2022},
  url = {https://openreview.net/forum?id=DvLUHcHRBj},
}
Figure