GRAPEVNE - Graphical Analytical Pipeline Development Environment for Infectious Diseases

John‐Stuart Brittain, Joseph L.-h. Tsui, Rhys Inward, Bernardo Gutiérrez, Gaspary Mwanyika, Houriiyah Tegally, Tuyen Huynh, George Githinji, Sofonias Kifle Tessema, John T. McCrone, Samir Bhatt, Abhishek Dasgupta, Stephen Ratcliffe, Moritz U. G. KraemerWellcome 2025 · DOI

TL;DR

GRAPEVNE is a new, open-source graphical platform that simplifies the creation and execution of complex data analysis pipelines for infectious disease research. By leveraging the Snakemake workflow system and providing an intuitive visual interface, it aims to make advanced computational analyses more accessible, reproducible, and collaborative, especially in resource-limited settings.

What Problem Does This Paper Address?

The explosion of data related to infectious diseases – from genomic sequences to epidemiological trends and spatial information – offers unprecedented opportunities for scientific discovery and public health decision-making. However, effectively utilizing this diverse and massive dataset is a significant challenge. Researchers often face hurdles due to:

  • Data heterogeneity: Differences in data preprocessing methods and formats across various sources.
  • Technical expertise: The need for specialized skills in data science, programming, and bioinformatics to build and run complex analyses.
  • Resource limitations: Unequal access to hardware, cloud computing, and skilled personnel, particularly in low-resource settings.
  • Reproducibility: Ensuring that analyses can be reliably repeated by others, which is crucial for scientific validation and public health trust.

These challenges hinder the ability to conduct large-scale, routine analyses, especially at a local level without the need to share sensitive data across borders.

Methods

GRAPEVNE tackles these problems by providing a Graphical Analytical Pipeline Development Environment. Its core components and methodologies include:

  • Snakemake Workflow Management: GRAPEVNE is built upon the Snakemake workflow management system. Snakemake allows for the definition of computational workflows as directed acyclic graphs (DAGs), where rules specify input-output transformations and execution order is determined by dependencies. This ensures scalability and fault tolerance.
  • Intuitive Graphical Interface: The platform offers a user-friendly, cross-platform graphical interface (built with Electron, Node.js, Python, Typescript, React, and Redux). This allows users to visually construct, configure, and manage complex analytical workflows by dragging and dropping modular components.
  • Modular Design: Workflows are built from self-contained modules, each representing a Snakemake workflow with its own configurations, scripts, and metadata. This modularity promotes interoperability and allows for the reuse and adaptation of components.
  • Wrapper System: GRAPEVNE introduces a dynamic wrapper system that provides an interface layer for modules, managing namespace redirection and compatibility checks. This allows users to rearrange module order dynamically without altering core dependencies, simplifying prototyping and iterative adjustments.
  • Dependency Management: Integration with Conda facilitates seamless management of software dependencies, enabling workflows to be executed in containerized environments for stringent reproducibility.
  • Community-Driven Ecosystem: The platform encourages community contributions through a searchable repository of modules (the 'vneyard'), fostering collaboration and the organic evolution of the tool.

Datasets

The paper demonstrates GRAPEVNE's utility through two case studies, utilizing publicly available datasets:

  • SARS-CoV-2 Variants of Concern (VOCs): This case study uses genomic data from GISAID (Global Initiative on Sharing All Influenza Data) and epidemiological data from sources like Our World in Data. The goal was to reconstruct the spatiotemporal spread of VOCs, leveraging air traffic network data.
  • Dengue Virus (DENV) Genomic Epidemiological Pipeline: This case study utilizes genomic data from GISAID EpiArbo and GenBank, along with local datasets. The aim is to track the spatio-temporal dynamics of DENV lineages.

The paper does not introduce new benchmark datasets but rather showcases the application of GRAPEVNE to existing, real-world infectious disease data.

Key Insights and Ideas

  • Bridging the Gap: GRAPEVNE effectively bridges the gap between complex computational tools and researchers with varying technical expertise, democratizing access to advanced infectious disease analysis.
  • Reproducibility as a Core Feature: By building on Snakemake and incorporating containerization, GRAPEVNE places a strong emphasis on ensuring that analyses are reproducible, a critical aspect for scientific rigor and public health trust.
  • Facilitating Interdisciplinary Collaboration: The platform's modular and visual nature, combined with its open-source ethos, is designed to foster collaboration between different scientific communities (e.g., epidemiologists, bioinformaticians, data scientists).
  • Empowering Local Analysis: The ability to run analyses locally without sharing sensitive data across borders is a significant advantage for public health institutions and researchers in diverse geographical contexts.
  • Extensibility and Adaptability: The modular design and wrapper system allow users to easily adapt existing pipelines or build new ones, catering to specific research questions and evolving scientific needs.

Key Results

The paper demonstrates GRAPEVNE's effectiveness through two detailed case studies:

  • SARS-CoV-2 VOC Pipeline: GRAPEVNE successfully reimplemented a complex phylogeographic analysis of VOC spread, which previously required coordinating multiple third-party software packages and custom scripts. This demonstrated GRAPEVNE's ability to streamline such multi-step, computationally intensive analyses.
  • Dengue Pipeline: A modular pipeline was developed for Dengue virus surveillance, capable of harmonizing data, assigning genotypes, aligning sequences, and performing various downstream analyses (phylogenetic, Bayesian, hypothesis testing). This pipeline can be run end-to-end, selectively, or expanded, showcasing GRAPEVNE's flexibility.

While specific quantitative results like speedup or accuracy improvements are not the primary focus, the qualitative results highlight GRAPEVNE's success in simplifying complex workflows, enhancing reproducibility, and facilitating data-driven insights for infectious disease research.

Limitations and Future Work

The authors acknowledge and suggest future directions:

  • Installation and Compatibility Issues: One reviewer noted encountering compatibility issues with macOS Ventura, indicating potential areas for improvement in installation and platform stability.
  • Direct Access to Replication Materials: A suggestion was made to provide more direct links to walkthroughs and datasets for case studies to further aid readers in reproducing workflows.
  • Validation of Community Modules: Clarification is needed on the process for validating community-contributed modules to ensure their quality, compatibility, and reliability.
  • Community Contributions: The authors actively encourage contributions from the wider data science, bioinformatics, and epidemiological community to translate their packages and pipelines into GRAPEVNE modules, aiming to expand the platform's capabilities.
  • Enhancing Epidemic Preparedness: The overarching goal is to leverage GRAPEVNE to improve the efficiency of processing, analysis, and sharing of large genomic datasets, thereby accelerating responses to infectious disease outbreaks and enhancing global epidemic preparedness.

Summary updated

Abstract

The increase in volume and diversity of relevant data on infectious diseases and their drivers provides opportunities to generate new scientific insights that can support 'real-time' decision-making in public health across outbreak contexts and enhance pandemic preparedness. However, utilising the wide array of clinical, genomic, epidemiological, and spatial data collected globally is difficult due to differences in data preprocessing, data science capacity, and access to hardware and cloud resources. To facilitate large-scale and routine analyses of infectious disease data at the local level (i.e. without sharing data across borders), we developed GRAPEVNE (Graphical Analytical Pipeline Development Environment), a platform enabling the construction of modular pipelines designed for complex and repetitive data analysis workflows through an intuitive graphical interface. Built on the <i>Snakemake</i> workflow management system, GRAPEVNE streamlines the creation, execution, and sharing of analytical pipelines. Its modular approach already supports a diverse range of scientific applications, including genomic analysis, epidemiological modeling, and large-scale data processing. Each module in GRAPEVNE is a self-contained Snakemake workflow, complete with configurations, scripts, and metadata, enabling interoperability. The platform's open-source nature ensures ongoing community-driven development and scalability. GRAPEVNE empowers researchers and public health institutions by simplifying complex analytical workflows, fostering data-driven discovery, and enhancing reproducibility in computational research. Its user-driven ecosystem encourages continuous innovation in biomedical and epidemiological research but is applicable beyond that. Key use-cases include automated phylogenetic analysis of viral sequences, real-time outbreak monitoring, forecasting, and epidemiological data processing. For instance, our dengue virus pipeline demonstrates end-to-end automation from sequence retrieval to phylogeographic inference, leveraging established bioinformatics tools which can be deployed to any geographical context. For more details, see documentation at: https://grapevne.readthedocs.io.
BibTeX
@inproceedings{brittain2025grapevne,
  author = {John‐Stuart Brittain and Joseph L.-H. Tsui and Rhys Inward and Bernardo Gutiérrez and Gaspary Mwanyika and Houriiyah Tegally and Tuyen Huynh and George Githinji and Sofonias Kifle Tessema and John T. McCrone and Samir Bhatt and Abhishek Dasgupta and Stephen Ratcliffe and Moritz U. G. Kraemer},
  title = {{GRAPEVNE - Graphical Analytical Pipeline Development Environment for Infectious Diseases}},
  booktitle = {Wellcome},
  year = {2025},
  doi = {10.12688/wellcomeopenres.23824.1},
}
Figure