The ENCODE Uniform Analysis Pipelines

Benjamin C. Hitz, Jin-Wook Lee, Otto Jolanki, Meenakshi S. Kagda, Keenan Graham, Paul Sud, Idan Gabdank, J. Seth Strattan, Cricket A. Sloan, Timothy R. Dreszer, Laurence D. Rowe, Nikhil R. Podduturi, Venkat S. Malladi, Esther T. Chan, Jean M. Davidson, Marcus Ho, Stuart R. Miyasato, Matt Simison, Forrest Y. Tanaka, Yunhai Luo, Ian Whaling, Eurie L. Hong, Brian T. Lee, Richard Sandstrom, Eric Rynes, Jemma Nelson, Andrew Nishida, Alyssa Ingersoll, Michael Buckley, Mark Frerker, Daniel Sunwook Kim, Nathan Boley, Diane Trout, Alexander Dobin, Sorena Rahmanian, Dana Wyman, Gabriela Balderrama-Gutierrez, Fairlie Reese, Neva C. Durand, Olga Dudchenko, David Weisz, Suhas S.p. Rao, Alyssa Blackburn, Dimos Gkountaroulis, Mahdi Sadr, Moshe Olshansky, Yossi Eliaz, Dat Nguyen, Ivan D. Bochkov, Muhammad S. Shamim, Ragini Mahajan, Erez Lieberman Aiden, T Gingeras, Simon Heath, Martin Hirst, W. James Kent, Anshul Kundaje, A Mortazavi, B Wold, J. Michael CherrybioRxiv 2023 · DOI

TL;DR

The ENCODE project has developed a suite of standardized, reproducible, and publicly available analysis pipelines for a wide range of genomic assays. These pipelines, built using Docker and WDL, ensure data consistency and interoperability across thousands of experiments, enabling researchers to access and analyze complex genomic data more efficiently.

What Problem Does This Paper Address?

The ENCODE project generates a massive amount of genomic data from diverse experiments and cell types. A major challenge in such large-scale genomics initiatives is ensuring that the data is processed consistently and reproducibly. Without standardized analysis methods, comparing results across different experiments or integrating data from various sources becomes incredibly difficult, hindering scientific discovery. This paper addresses this by detailing the development and implementation of uniform analysis pipelines that standardize the processing of raw sequencing data, ensuring data provenance, reproducibility, and interoperability.

Methods

The core of the ENCODE uniform analysis pipelines relies on several key technologies:

  • Docker: For containerizing software and dependencies, ensuring that pipelines run consistently across different computing environments.
  • Workflow Description Language (WDL): A domain-specific language for defining and executing computational workflows, allowing for the description of each analysis step, its inputs, outputs, and resource requirements.
  • Cromwell: A workflow execution engine that manages the execution of WDL workflows across various platforms, including cloud environments (Google Cloud, AWS) and local HPC clusters.
  • GitHub: For hosting the open-source code of all pipelines, enabling collaboration and community contributions.
  • Dockerhub: For distributing containerized pipeline images.
  • CAPER and CROO: Custom software packages developed to facilitate efficient workflow submission and output organization, respectively.

The paper details specific pipelines for various assays, including ChIP-seq, ATAC-seq, RNA-seq (bulk, microRNA, long-read), DNase-seq, WGBS, and Hi-C. Each pipeline is tailored to the specific requirements of the assay, employing appropriate mapping, peak calling, quantification, and quality control tools.

Datasets

The paper describes the ENCODE project's comprehensive dataset, which includes:

  • Over 19,000 functional genomics experiments.
  • Data from more than 1,000 cell lines and tissues.
  • Experiments covering Homo sapiens and Mus musculus genomes.
  • Data from additional species like D. melanogaster and C. elegans (via modENCODE).
  • Incorporation of over 2,000 datasets from the Roadmap for Epigenomics Consortium (REMC) and the Genomics of Gene Regulation (GGR) project.
  • The pipelines are designed to process a vast number of raw sequencing files (e.g., over 80,000 fastq files for ENCODE).

The paper does not introduce new benchmark datasets but rather focuses on the standardization of analysis for existing and ongoing ENCODE data.

Key Insights and Ideas

  • Standardization is Crucial for Reproducibility: The paper strongly emphasizes that uniform pipelines are essential for ensuring that results are comparable and reproducible across the vast and diverse ENCODE dataset.
  • Cloud Computing Democratizes Access: By leveraging cloud platforms and containerization, ENCODE pipelines allow even small labs without extensive computational infrastructure to access and analyze complex genomic data.
  • Metadata and Provenance are Paramount: The ENCODE project meticulously captures metadata, including software versions, parameters, and quality metrics, for every step of the pipeline. This ensures full data provenance and allows users to understand exactly how a result was generated.
  • Open Source and Community Collaboration: Making the pipeline code publicly available on GitHub fosters transparency, allows for community contributions, and promotes wider adoption and adaptation of these tools.
  • Pipeline Infrastructure is a Monumental Engineering Task: The development and maintenance of these robust, scalable, and reproducible pipelines represent a significant engineering achievement, requiring careful design and continuous refinement.

Key Results

The paper details the implementation of uniform pipelines for numerous genomic assays:

  • ChIP-seq: Pipelines for transcription factor (TF) and histone ChIP-seq, handling both narrow and broad peaks, and adapted for multiplexed MINT-Chip assays.
  • RNA-seq: Five distinct pipelines for various transcript types (short, long-read, microRNA), polyadenylation status, and subcellular localization.
  • ATAC-seq, DNase-seq, Hi-C, WGBS: Pipelines for these assays are also described, each with specific adaptations for their unique experimental designs and analysis requirements.
  • Performance: Pipelines are designed to be efficient, with typical execution times ranging from minutes to hours per million reads, and requiring manageable amounts of RAM. For example, the DNase-seq pipeline averages 1.3 CPU hours per million reads with a 32GB memory footprint.
  • Data Availability: All processed data and pipeline code are made available through the ENCODE portal and cloud platforms like AWS and Azure.

Limitations and Future Work

  • Complexity of ChIP-seq: The paper notes that ChIP-seq, particularly TF ChIP-seq, is more complicated than other assays due to inherent variability in signal-to-noise ratio and peak detection, requiring sophisticated quality control and reproducibility metrics like IDR.
  • Continuous Evolution: The pipelines are continuously modified as the ENCODE project progresses and new data is analyzed, implying that they are not static but rather evolving systems.
  • Future Work: The paper implicitly suggests that the ongoing development and maintenance of these pipelines, along with the integration of new assays and technologies, will continue to be a focus. The emphasis on making steps idempotent and the use of continuous integration (CI) services point towards a commitment to robust engineering and future adaptability.

Summary updated

Abstract

The Encyclopedia of DNA elements (ENCODE) project is a collaborative effort to create a comprehensive catalog of functional elements in the human genome. The current database comprises more than 19000 functional genomics experiments across more than 1000 cell lines and tissues using a wide array of experimental techniques to study the chromatin structure, regulatory and transcriptional landscape of the <i>Homo sapiens</i> and <i>Mus musculus</i> genomes. All experimental data, metadata, and associated computational analyses created by the ENCODE consortium are submitted to the Data Coordination Center (DCC) for validation, tracking, storage, and distribution to community resources and the scientific community. The ENCODE project has engineered and distributed uniform processing pipelines in order to promote data provenance and reproducibility as well as allow interoperability between genomic resources and other consortia. All data files, reference genome versions, software versions, and parameters used by the pipelines are captured and available <i>via</i> the ENCODE Portal. The pipeline code, developed using Docker and Workflow Description Language (WDL; https://openwdl.org/) is publicly available in GitHub, with images available on Dockerhub (https://hub.docker.com), enabling access to a diverse range of biomedical researchers. ENCODE pipelines maintained and used by the DCC can be installed to run on personal computers, local HPC clusters, or in cloud computing environments <i>via</i> Cromwell. Access to the pipelines and data <i>via</i> the cloud allows small labs the ability to use the data or software without access to institutional compute clusters. Standardization of the computational methodologies for analysis and quality control leads to comparable results from different ENCODE collections - a prerequisite for successful integrative analyses.
BibTeX
@misc{hitz2023encode,
  author = {Benjamin C. Hitz and Jin-Wook Lee and Otto Jolanki and Meenakshi S. Kagda and Keenan Graham and Paul Sud and Idan Gabdank and J. Seth Strattan and Cricket A. Sloan and Timothy R. Dreszer and Laurence D. Rowe and Nikhil R. Podduturi and Venkat S. Malladi and Esther T. Chan and Jean M. Davidson and Marcus Ho and Stuart R. Miyasato and Matt Simison and Forrest Y. Tanaka and Yunhai Luo and Ian Whaling and Eurie L. Hong and Brian T. Lee and Richard Sandstrom and Eric Rynes and Jemma Nelson and Andrew Nishida and Alyssa Ingersoll and Michael Buckley and Mark Frerker and Daniel Sunwook Kim and Nathan Boley and Diane Trout and Alexander Dobin and Sorena Rahmanian and Dana Wyman and Gabriela Balderrama-Gutierrez and Fairlie Reese and Neva C. Durand and Olga Dudchenko and David Weisz and Suhas S.P. Rao and Alyssa Blackburn and Dimos Gkountaroulis and Mahdi Sadr and Moshe Olshansky and Yossi Eliaz and Dat Nguyen and Ivan D. Bochkov and Muhammad S. Shamim and Ragini Mahajan and Erez Lieberman Aiden and T Gingeras and Simon Heath and Martin Hirst and W. James Kent and Anshul Kundaje and A Mortazavi and B Wold and J. Michael Cherry},
  title = {{The ENCODE Uniform Analysis Pipelines}},
  year = {2023},
  doi = {10.1101/2023.04.04.535623},
  howpublished = {bioRxiv},
}
Figure