Heterogeneous Datasets for Federated Survival Analysis Simulation

Alberto Archetti, Eugenio Lomurno, Francesco Lattari, André Martin, Matteo Matteucci

TL;DR

This paper introduces a reproducible framework for generating heterogeneous federated datasets from existing centralized survival analysis data. By using Dirichlet-based splitting algorithms, researchers can now precisely control data imbalance and distribution shifts among simulated clients, providing a much-needed benchmarking tool for developing privacy-preserving survival models.

What Problem Does This Paper Address?

Survival analysis—predicting the time until an event like a mechanical failure or a medical complication—is critical in fields where data is often siloed and highly confidential. While Federated Learning (FL) offers a way to train models on this distributed data without compromising privacy, the field lacks standardized, realistic datasets for testing.

In the real world, data isn't "Identically and Independently Distributed" (IID). One hospital might have thousands of patients (quantity skew), while another might specialize in high-risk cases with much shorter survival times (label skew). Most existing FL simulations use uniform data splits that don't reflect these messy realities. Without a way to simulate this "heterogeneity," we can't know if our federated survival models will actually work when deployed in diverse, real-world environments.

Methods

The authors propose two novel splitting algorithms based on the Dirichlet distribution, which acts as a mathematical "knob" to control how skewed the data becomes.

  1. Quantity-Skewed Splitting: This assigns a different number of samples to each client. By tuning a similarity parameter ($\alpha$), researchers can simulate a federation where some clients are "data-rich" and others are "data-poor."
  2. Label-Skewed Splitting: This is more complex. The survival timeline is divided into discrete "bins." The algorithm then ensures that different clients receive different proportions of samples from these bins. For example, one client might end up with mostly "early-event" samples, while another gets "late-event" or "censored" samples.

The beauty of this approach is that a single hyperparameter ($\alpha$) allows researchers to transition smoothly from a perfectly uniform distribution ($\alpha \to \infty$) to extreme heterogeneity ($\alpha \to 0$).

Datasets

The study applies these splitting techniques to five well-known centralized survival datasets, effectively turning them into federated benchmarks:

  • GBSG2: Breast cancer study data.
  • METABRIC: Molecular taxonomy of breast cancer.
  • AIDS: Clinical trial data for HIV treatment.
  • FLCHAIN: Assay of serum free light chain.
  • SUPPORT: A large study on hospitalized adults' prognoses.

These datasets vary significantly in size (from 686 to over 9,000 samples) and censoring rates (28% to 68%), providing a diverse testing ground.

Key Insights and Ideas

  • Quantity Skew $\neq$ Distribution Skew: One of the most important findings is that simply giving clients different amounts of data (quantity skew) does not significantly change the underlying survival curves. The "heterogeneity score" remained below 5% in these cases. This means quantity skew tests a model's resilience to imbalance, but not to population differences.
  • The Power of Time-Binning: By treating survival times as "classes" through binning, the authors successfully adapted Dirichlet-based partitioning—usually used for image classification—to the continuous time-domain of survival analysis.
  • Quantifying "Difference": The authors didn't just look at the splits; they used log-rank tests to statistically prove that their label-skewing algorithm creates populations that are truly different from one another.

Key Results

The authors introduced a heterogeneity score ($h$), representing the percentage of client pairs that have statistically significant differences in their survival distributions.

  • Label-Skewed Success: For the SUPPORT dataset with 5 clients, the $h$ score jumped from 0.5% (at $\alpha=1000$, near-uniform) to 94.0% (at $\alpha=0.1$, highly heterogeneous).
  • Visual Evidence: Kaplan-Meier plots (which visualize survival probability over time) clearly showed the curves for different clients diverging as $\alpha$ decreased, mimicking real-world silos where patient outcomes vary by location or demographic.
  • Reproducibility: The implementation is publicly available, allowing other researchers to generate the exact same splits for their own model comparisons.

Limitations and Future Work

While this paper provides a robust framework for data heterogeneity, it focuses primarily on right-censoring (where we know an event hasn't happened yet, but not when it will). Future work could extend these algorithms to handle left-censoring or interval-censoring.

Additionally, the paper focuses on data heterogeneity. In real-world FL, system heterogeneity (differences in client hardware, battery life, and network speed) is also a major factor. Combining this data-splitting framework with system-level simulation would be the "gold standard" for future federated survival analysis research.

Summary updated

Abstract

Survival analysis studies time-modeling techniques for an event of interest occurring for a population. Survival analysis found widespread applications in healthcare, engineering, and social sciences. However, the data needed to train survival models are often distributed, incomplete, censored, and confidential. In this context, federated learning can be exploited to tremendously improve the quality of the models trained on distributed data while preserving user privacy. However, federated survival analysis is still in its early development, and there is no common benchmarking dataset to test federated survival models. This work provides a novel technique for constructing realistic heterogeneous datasets by starting from existing non-federated datasets in a reproducible way. Specifically, we propose two dataset-splitting algorithms based on the Dirichlet distribution to assign each data sample to a carefully chosen client: quantity-skewed splitting and label-skewed splitting. Furthermore, these algorithms allow for obtaining different levels of heterogeneity by changing a single hyperparameter. Finally, numerical experiments provide a quantitative evaluation of the heterogeneity level using log-rank tests and a qualitative analysis of the generated splits. The implementation of the proposed methods is publicly available in favor of reproducibility and to encourage common practices to simulate federated environments for survival analysis.
BibTeX
@inproceedings{archetti2023heterogeneous,
  author = {Alberto Archetti and Eugenio Lomurno and Francesco Lattari and André Martin and Matteo Matteucci},
  title = {{Heterogeneous Datasets for Federated Survival Analysis Simulation}},
  booktitle = {ICPE (Companion)},
  year = {2023},
  doi = {10.1145/3578245.3584935},
}

Loading related content...

Figure