TL;DR
This paper introduces workflowr, an R package designed to make scientific research code more reproducible and shareable. It achieves this by integrating version control (Git), literate programming (R Markdown), automatic checks, and website generation, simplifying a complex process for researchers.
What Problem Does This Paper Address?
Reproducibility is a cornerstone of scientific progress, yet it's often a challenge in practice. Many scientific analyses rely heavily on code, but this code is frequently not shared or is difficult to run due to issues with organization, documentation, and the use of complex computational tools. This makes it hard for other researchers to verify, extend, or build upon existing work. The paper highlights that even with powerful interactive environments like R, it's easy to generate results that are difficult to reproduce due to unrecorded random seeds or "contaminated" computational environments. Furthermore, sharing code via platforms like GitHub requires familiarity with technologies that many scientists lack.
Methods
workflowr tackles these challenges by providing a structured "workflow" that integrates several key technologies:
- Version Control (Git):
workflowruses Git to track the development history of a project, allowing users to easily revert to previous versions of code and results. It simplifies Git usage for researchers who may not be Git experts. - Literate Programming (R Markdown): Analyses are written in R Markdown files, which combine code, narrative text, and output. This ensures that the code and its results are presented together in a readable format.
workflowrbuilds uponknitrandrmarkdownfor this purpose. - Automatic Checks and Safeguards: The
wflow_build()function automatically performs several checks to enhance reproducibility. These include: - Running code in a clean R session.
- Ensuring consistent working directory settings.
- Setting a seed for the pseudorandom number generator.
-
Recording information about the computing environment (OS, R version, packages used). These checks are summarized in a reproducibility report displayed on the generated website.
-
Sharing via Website:
workflowrautomatically generates a browsable website for each project, displaying results, figures, and the development history. This makes it easy to share findings with collaborators by simply providing a URL.
The package also integrates with the RStudio IDE and uses the git2r package to interface with Git.
Datasets
The paper does not describe specific datasets used for training or evaluation of the workflowr package itself. Instead, it focuses on the methodology and how workflowr can be applied to any research project that generates results using R code. The "Use cases" section mentions several example projects that utilize workflowr with their respective datasets, but these are illustrative examples rather than formal benchmarks.
Key Insights and Ideas
- Democratizing Reproducibility:
workflowraims to make reproducible research accessible to all scientists, regardless of their computational background, by abstracting away much of the complexity of tools like Git and R Markdown. - Integrated Workflow: The power of
workflowrlies in its seamless integration of version control, literate programming, and website generation. This creates a cohesive system where code, results, and documentation are intrinsically linked. - "Workflow" as a Guiding Principle: The package instills a specific "workflow" – a repeatable sequence of steps – that promotes good practices for reproducible research from the outset of a project.
- Website as a Living Document: The automatically generated website serves not just as a static report but as a dynamic record of the project's evolution, complete with links to past versions of analyses and figures.
Key Results
The paper demonstrates the functionality of workflowr through its core commands (wflow_start, wflow_build, wflow_publish, wflow_status, wflow_git_push) and provides examples of its application. The key "result" is the creation of a reproducible research project structure that can be easily shared. The paper highlights that workflowr has been downloaded over 7,000 times from CRAN since its release in April 2018, indicating its adoption and utility within the research community.
Limitations and Future Work
The authors acknowledge several limitations and suggest future directions:
- Large Files:
workflowris not well-suited for tracking very large files, which often need to be excluded from the Git repository. A workaround using Git LFS is mentioned but requires significant expertise. - Git Expertise: While
workflowrsimplifies Git, users with existing Git experience might inadvertently break the workflow by using advanced Git commands. - Environment Replication:
workflowrrecords the computing environment but doesn't provide tools for replicating it. The authors suggest complementary tools like Docker, Conda, and packrat for this purpose and envision future integration. - Centralized Project Website: The idea of a centralized website for discovering
workflowrprojects is proposed. - Dependency Tracking: Future work aims to implement syntax for recording file dependencies within Rmd files and incorporating dependency checking into reproducibility safeguards.
- Hosting Options: Discussion of options for private/protected hosting and alternative hosting services beyond
raw.githack.comwould be valuable. - Comparison to Alternatives: A comparison with other tools like
drakeis suggested. - Reproducibility Pitfalls: More in-depth discussion of potential pitfalls and guidance on issues like merge conflicts or handling large datasets would be beneficial.
Summary updated