dsLassoCov: a federated machine learning approach incorporating covariate control

Han Cao, Augusto Anguita, Charline Warembourg, Xavier Escriba-Montagut, Martine Vrijheid, Juan R. González, Tim Cadman, Verena Schneider-Lindner, Daniel Durstewitz, Xavier Basagaña, Emanuel SchwarzarXiv 2024 · DOI · arXiv

Abstract

In biomedical research, the integration of multi-site datasets is often hindered by legal restrictions and privacy concerns, leading to the adoption of Federated Learning (FL). However, existing FL frameworks frequently struggle with statistical heterogeneity and the presence of confounding variables, particularly in high-dimensional contexts such as genomics or exposomics. This paper presents dsLassoCov, a novel federated approach that incorporates covariate adjustment directly into the LASSO optimization process. By utilizing an iterative debiasing mechanism and Nesterov’s acceleration, dsLassoCov identifies true biomarkers across distributed databases while maintaining strict data privacy. Our results, validated through both simulated stress tests and real-world exposome data from the HELIX project, demonstrate superior performance in classification tasks and significant computational efficiency compared to traditional federated baselines.

Introduction

The paradigm of "more data leads to better models" is a cornerstone of modern bioinformatics. Yet, much of this data remains trapped in institutional silos due to regulations like the GDPR. Federated Learning (FL) has emerged as a solution, allowing models to be trained locally at various sites with only model parameters being shared centrally.

Despite its promise, FL faces a significant hurdle: statistical heterogeneity. Differences in data collection protocols and biological factors—such as age, sex, or ancestry—act as confounders. These variables can create spurious correlations between features and outcomes, leading to biased models. While centralized studies can easily "regress out" these variables, doing so in a federated, high-dimensional setting is computationally intensive and often fails for classification tasks.

Methodology: The dsLassoCov Framework

To address these challenges, we developed dsLassoCov (DataSHIELD LASSO adjusted for Covariates), built upon the DataSHIELD infrastructure—an R-based framework designed for privacy-preserving distributed analysis.

Integrated Optimization and Iterative Debiasing

Unlike traditional two-step methods that attempt to "clean" data before training, dsLassoCov integrates covariates ($w^{(c)}$) directly into the objective function. The algorithm employs a mathematical approach equivalent to "double machine learning." In each iteration, the association of covariates is implicitly removed from both the features and the outcome. This ensures that the resulting model weights are "covariate-free," effectively isolating the true signal from the confounding noise.

Computational Efficiency

Communication overhead is a primary bottleneck in federated systems. To mitigate this, dsLassoCov utilizes Nesterov’s acceleration, which speeds up the convergence of the optimization process. Furthermore, while traditional federated methods might require regressing out covariates feature-by-feature—a process that becomes untenable with thousands of genes—dsLassoCov handles this adjustment "all-at-once" within the main training loop, drastically reducing the number of required communication rounds.

Experimental Validation

Simulation Studies

We conducted stress tests using simulated data involving up to 3,000 features across 10 simulated servers. These tests allowed us to manipulate the "intensity" of confounding effects. The results indicated that confounding effects on the outcome (alpha noise) are significantly more detrimental to model accuracy than effects on the features (gamma noise). dsLassoCov remained stable and accurate even under high levels of alpha noise, where baseline methods failed.

Real-World Application: The HELIX Project

To validate the tool in a clinical context, we applied dsLassoCov to data from the Human Early-Life Exposome (HELIX) project. This dataset comprised six European birth cohorts, including 1,298 children and 77 environmental exposures (e.g., air pollution, chemical levels). We controlled for 13 confounders, including maternal age and education, to identify exposures linked to childhood hypertension.

dsLassoCov successfully identified 32 exposures associated with hypertension. Notably, factors such as building density and PFAS exposure were identified as risks, while access to public transport was found to be protective. These results perfectly replicated findings from previous non-federated studies, confirming that dsLassoCov can achieve centralized-level accuracy in a distributed, privacy-preserving environment.

Key Results and Discussion

  1. Classification Performance: In high-dimensional regression and classification tests, dsLassoCov consistently outperformed meta-analysis and standard federated approaches. In scenarios where other methods saw significant drops in feature selection accuracy, dsLassoCov maintained a robust ~60% accuracy rate.
  2. The Classification Edge: A critical finding was that traditional "residualization" (removing confounders from the outcome before training) is often ineffective for binary classification because continuous confounders cannot be easily subtracted from a binary (Yes/No) outcome. dsLassoCov’s integrated approach bypasses this limitation entirely.
  3. Scalability: The implementation proved significantly faster than the conventional federated baseline (GLM + LASSO sequence) in high-dimensional scenarios, making it a viable tool for large-scale multi-omic studies.

Limitations and Future Work

While dsLassoCov represents a significant step forward, it is currently limited to linear LASSO models. As many biological associations may be non-linear, extending this framework to non-linear models is a priority for future development. Additionally, the current implementation is optimized for "cross-silo" FL (collaboration between a few large institutions) rather than "cross-device" FL (thousands of individual devices). Future iterations will explore more complex model architectures while maintaining the rigorous covariate removal mechanism that defines the dsLassoCov approach.

Summary updated

BibTeX
@misc{cao2024dslassocov,
  author = {Han Cao and Augusto Anguita and Charline Warembourg and Xavier Escriba-Montagut and Martine Vrijheid and Juan R. González and Tim Cadman and Verena Schneider-Lindner and Daniel Durstewitz and Xavier Basagaña and Emanuel Schwarz},
  title = {{dsLassoCov: a federated machine learning approach incorporating covariate control}},
  year = {2024},
  eprint = {2412.07991},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2412.07991},
}
Figure