Exploring the Potential of Carbon-Aware Execution for Scientific Workflows

Kathleen West, Fabian Lehmann, Vasilis Bountris, Ulf Leser, Yehia Elkhatib, Lauritz Thamsen

Abstract

Scientific workflows are significant consumers of computational resources and contributors to carbon emissions. This study investigates the potential of "carbon-aware" execution—leveraging workflow flexibility to align computation with periods of low grid carbon intensity. By simulating interventions such as temporal shifting, task interruption, and hardware scaling on real-world genomics and earth observation pipelines, we demonstrate that carbon footprints can be reduced by up to 66% with minimal impact on completion times.

Introduction

Scientific research increasingly relies on complex computational pipelines for data-intensive tasks. These workflows often run for extended periods on large clusters, leading to substantial energy consumption. Carbon-aware computing offers a pathway to sustainability by scheduling workloads when renewable energy is most prevalent on the grid. Scientific workflows are uniquely suited for this approach due to their delay tolerance, interruptibility, and scalability. This paper seeks to quantify the potential carbon savings achievable through intelligent scheduling and resource management.

Methodology

The study utilizes Ichnos, a tool that estimates carbon footprints by applying linear power models to historical execution traces. These traces are mapped to real-world Carbon Intensity (CI) data, considering both average and marginal emissions. The researchers evaluated two workflows from the nf-core library: Chip-Seq (genomics) and Rangeland (earth observation).

Three primary interventions were simulated:

  1. Entire Workflow Shifting: Delaying the start of a workflow to coincide with greener energy windows.
  2. Interrupted Workflow Shifting: Pausing execution between tasks to avoid carbon intensity spikes.
  3. Resource and Hardware Scaling: Adjusting CPU frequencies and node counts to "shape" the workload to fit within low-carbon windows.
Figure 2: Strategic resource assignment allows tasks to be scheduled during periods of lower carbon intensity.

Key Insights

A critical finding is that scientific workflows are naturally interruptible. Unlike monolithic applications, workflows save intermediate data to disk, allowing them to be paused and resumed with negligible overhead. This flexibility enables "interrupted shifting," where tasks are executed across multiple non-consecutive green windows, often yielding higher savings than simple start-time delays. Furthermore, workload "shaping" through scaling allows researchers to compress execution times to fit within brief windows of highly renewable energy without necessarily increasing total energy consumption.

Results

The baseline carbon footprint for the Chip-Seq workflow was approximately 27.6 kgCO2e, while Rangeland generated roughly 10.0 kgCO2e. Implementing a 192-hour shifting window for Chip-Seq resulted in a dramatic reduction of up to 66% in marginal CI emissions. For the Rangeland workflow, interrupted shifting achieved a 52% reduction with only a 3% increase in total runtime.

Figure 5: Percentage reduction in carbon emissions for Chip-Seq and Rangeland workflows across different scheduling windows for both Average and Marginal Carbon Intensity.

Hardware scaling also proved effective; increasing the node count for Chip-Seq from 2 to 8 reduced the carbon footprint by 42.5% by enabling the job to complete within a specific low-carbon window.

Conclusion and Future Work

This research establishes an upper bound for carbon savings in scientific computing, assuming perfect grid forecasts and task duration knowledge. Future efforts should focus on developing dynamic, real-time carbon-aware schedulers capable of handling the uncertainties of grid fluctuations and task variability. While this study focuses on operational carbon, future assessments should also incorporate the embodied carbon of the underlying hardware to provide a holistic view of computational sustainability.

Summary updated

Abstract

Scientific workflows are widely used to automate scientific data analysis and often involve processing large quantities of data on compute clusters. As such, their execution tends to be long-running and resource intensive, leading to significant energy consumption and carbon emissions. Meanwhile, a wealth of carbon-aware computing methods have been proposed, yet little work has focused specifically on scientific workflows, even though they present a substantial opportunity for carbon-aware computing because they are inherently delay tolerant, efficiently interruptible, and highly scalable. In this study, we demonstrate the potential for carbonaware workflow execution. For this, we estimate the carbon footprint of two real-world Nextflow workflows executed on cluster infrastructure. We use a linear power model for energy consumption estimates and real-world average and marginal CI data for two regions. We evaluate the impact of carbonaware temporal shifting, pausing and resuming, and resource scaling. Our findings highlight significant potential for reducing emissions of workflows and workflow tasks.
BibTeX
@misc{west2025exploring,
  author = {Kathleen West and Fabian Lehmann and Vasilis Bountris and Ulf Leser and Yehia Elkhatib and Lauritz Thamsen},
  title = {{Exploring the Potential of Carbon-Aware Execution for Scientific Workflows}},
  year = {2025},
  eprint = {2503.13705},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2503.13705},
}
Figure