Embedding the de.NBI Cloud in the National Research Data Infrastructure Activities

Nils Hoffmann, Irena Maus, Sebastian Beier, Peter Belmann, Jan Krüger, Andreas Tauch, Alexander Goesmann, Roland Eils, Peer Bork, Oliver Kohlbacher, Ursula Kummer, Rolf Backofen, Ivo Buchhalter, Alexander SczyrbaCoRDI 2023 · DOI

Introduction

Modern life sciences are currently navigating a significant data explosion. Advances in high-throughput sequencing and biological imaging have led to an exponential increase in experimental output. However, many research groups lack the local computing power or technical expertise to process this "Big Data" effectively. The challenge is twofold: providing the hardware required for massive parallelization and creating a secure, unified environment for cross-institutional collaboration. The de.NBI Cloud addresses these challenges by serving as a federated high-performance computing infrastructure designed to bridge the gap between local research and national-scale infrastructure.

Infrastructure and Methodology

The de.NBI Cloud operates on a federated model, linking eight distributed centers across Germany—including locations in Berlin, Bielefeld, and Heidelberg—under a unified governance model. This approach demonstrates that a distributed network can provide a robust, scalable backbone when governed by common standards.

To accommodate varying levels of technical expertise, the cloud offers a tiered access model:

  • OpenStack: Provides "power users" with full control over virtual networks and storage.
  • Kubernetes (K8s): Facilitates the deployment of containerized, highly available applications.
  • SimpleVM: An in-house abstraction layer that allows researchers without a cloud background to launch pre-configured environments like RStudio or JupyterLab with minimal effort.
  • Galaxy: A web-based platform for graphical workflow management.

Security is managed through the LifeScience AAI, an EduGAIN-compatible single sign-on system, and centers maintain ISO27001 certification to ensure high standards for information security.

Scale and Scientific Impact

The de.NBI Cloud has established itself as a critical resource for the German research community. The scale of the infrastructure and its integration into the National Research Data Infrastructure (NFDI) landscape are summarized in the following overview:

Overview of the de.NBI Cloud infrastructure, including hardware capacity, user statistics, and integration with NFDI consortia.

The federation provides substantial hardware capacity, including 56,000 CPU cores, 240 TB of RAM, 100 PB of storage, and 520 GPUs. This capacity supports over 2,500 users across more than 850 projects, resulting in over 620 publications to date.

Integration and Collaborative Ecosystem

Beyond hardware, the de.NBI Cloud serves as a collaborative ecosystem. It hosts the Galaxy service and supports various NFDI consortia, such as NFDI4Microbiota, GHGA, and NFDI4Biodiversity. By providing standardized data processing agreements and centralized security training, the platform makes the federation legally and operationally viable for diverse scientific domains. It also connects directly to the European Life Science Infrastructure (ELIXIR), ensuring alignment with international standards.

Conclusion and Future Outlook

While the de.NBI Cloud has successfully democratized high-performance computing for life sciences, challenges remain in managing the complex legal requirements of data processing agreements. Future efforts will focus on deeper integration with the broader NFDI landscape, positioning the de.NBI Cloud as a template for other scientific domains needing to scale data-driven research across national ecosystems.

Summary updated

BibTeX
@inproceedings{hoffmann2023embedding,
  author = {Nils Hoffmann and Irena Maus and Sebastian Beier and Peter Belmann and Jan Krüger and Andreas Tauch and Alexander Goesmann and Roland Eils and Peer Bork and Oliver Kohlbacher and Ursula Kummer and Rolf Backofen and Ivo Buchhalter and Alexander Sczyrba},
  title = {{Embedding the de.NBI Cloud in the National Research Data Infrastructure Activities}},
  booktitle = {CoRDI},
  year = {2023},
  doi = {10.52825/cordi.v1i.387},
}
Figure