A Query Workflow Design to Perform Automatable Distributed Regression Analysis in Large Distributed Data Networks

Qoua L. Her, Jessica Malenfant, Sarah Malek, Yury Vilk, Jessica G. Young, Lingling Li, Jeffrey S. Brown, Sengwee ToheGEMs 2018 · DOI

TL;DR

This paper introduces a novel query workflow and process framework for automating distributed regression analysis (DRA) within large, distributed data networks (DDNs) using the PopMedNet™ platform. This advancement addresses the challenge of performing complex statistical analyses on sensitive patient data without direct data pooling, paving the way for more efficient and scalable health research.

What Problem Does This Paper Address?

Conducting large-scale health research often requires analyzing patient data from multiple sources. However, strict privacy regulations and data security concerns make it difficult to pool this sensitive information directly. Distributed Data Networks (DDNs) offer a solution by allowing data to remain at its source while analyses are performed collaboratively. While basic analyses can be done on aggregated data, complex statistical methods like regression analysis traditionally required patient-level data sharing. Distributed Regression Analysis (DRA) offers a privacy-preserving alternative by only exchanging intermediate statistical outputs, but its routine implementation in large DDNs has been hindered by the complex, iterative nature of the analysis and the need for extensive coordination. This paper tackles the challenge of making DRA automatable and routine within existing DDN infrastructure.

Methods

The authors developed a three-step process framework and a PopMedNet™ query workflow to enable automatable DRA. The core innovation lies in enhancing the PopMedNet infrastructure to support iterative, automated file transfers between data partners and an analysis center.

  1. Assemble De-identified Patient-Level Dataset: A distributed program is used to create a de-identified patient-level analytic dataset at each data partner site. This dataset remains behind the partner's firewall.
  2. Distribute DRA Package: A DRA package, containing SAS programs and initial regression parameter estimates, is distributed to data partners via PopMedNet™. Data partners execute this package locally on their de-identified datasets.
  3. Iterative File Transfer and Analysis: This is the most significant enhancement. The system now supports automated iterative transfer of intermediate statistics from data partners to the analysis center and updated regression parameter estimates back to the data partners. This is achieved through a new "DRA adapter" in PopMedNet™ that monitors for trigger text files, initiating automated uploads and downloads of files to pre-defined folders. This iterative process continues until the regression model converges or a maximum number of iterations is reached.

The workflow is designed to be agnostic to statistical software (though implemented with SAS) and accommodates various regression models (linear, logistic, Cox proportional hazards). It also allows for different levels of user-specified automation.

Datasets

The study was conducted within the FDA-funded Sentinel System, a large DDN comprising 18 data partners. The authors surveyed the hardware and software configurations of 16 of these partners to guide their design. No new datasets were introduced; rather, the work focused on developing a new analytical workflow for existing data within the Sentinel System.

Key Insights and Ideas

  • Automation is Key for Scalability: The iterative nature of DRA, requiring frequent exchange of intermediate statistics, is a major bottleneck. Automating this file transfer process is crucial for making DRA practical in large DDNs.
  • Leveraging Existing Infrastructure: The authors successfully integrated their DRA workflow into the existing PopMedNet™ platform and Sentinel System, minimizing the need for new software installations or significant changes to data partners' configurations. This is a critical factor for adoption.
  • Trigger Files as Orchestrators: The use of trigger text files to signal the completion of tasks and initiate subsequent automated processes is an elegant solution for coordinating the complex iterative workflow.
  • User Acceptance is Paramount: While the technical capability for full automation is developed, the authors acknowledge that user acceptance is vital. Many data partners prefer to review files before transfer, suggesting a phased approach to automation, starting with manual or semi-automated options to build trust.
  • Generalizability: The developed DRA capability is built into the core PopMedNet™ code, making it readily extendable to other PopMedNet-based DDNs like PCORnet and the NIH Collaboratory.

Key Results

The paper successfully developed and implemented a three-step process framework and PopMedNet™ query workflow that enables automatable DRA. This workflow has been piloted in the Sentinel System. The authors demonstrated that this approach can accommodate different regression models and levels of automation, and it requires minimal modifications to existing data partner configurations. While specific quantitative results on the performance of the DRA itself (e.g., convergence speed, accuracy compared to pooled analysis) are not the primary focus, the paper establishes the feasibility and methodology for its automatable implementation.

Limitations and Future Work

  • Computer System Failures/Interruptions: The current workflow does not optimally handle interruptions between DRA iterations. Future work will focus on enabling the process to pause or restart from the last error-free iteration.
  • Other Statistical Models and Software: While the workflow is agnostic to statistical software and accommodates common regression models, further development is needed for more complex statistical models and other software packages.
  • Vertically Partitioned Data: The current work focuses on horizontally partitioned data. Extending DRA to vertically partitioned data will require new analytic code, though the PopMedNet™ file transfer process can be reused.
  • User Acceptance and Automation Levels: The authors note mixed reactions from data partners regarding full automation. Future work may involve exploring other secure multiparty computation protocols to enhance data partner acceptability.
  • Diagnostic Statistics: The paper mentions the computation of diagnostic statistics upon model convergence, but the details of this process and its integration are not elaborated upon in this specific article.

Summary updated

Abstract

DRA has great potential to change the paradigm of data analysis in DDNs. Successful implementation of DRA in Sentinel will facilitate adoption of the analytic approach in other DDNs.
BibTeX
@inproceedings{her2018query,
  author = {Qoua L. Her and Jessica Malenfant and Sarah Malek and Yury Vilk and Jessica G. Young and Lingling Li and Jeffrey S. Brown and Sengwee Toh},
  title = {{A Query Workflow Design to Perform Automatable Distributed Regression Analysis in Large Distributed Data Networks}},
  booktitle = {eGEMs},
  year = {2018},
  doi = {10.5334/egems.209},
}
Figure