Abstract
Scientific workflows are significant consumers of computational resources and contributors to carbon emissions. This study investigates the potential of "carbon-aware" execution—leveraging workflow flexibility to align computation with periods of low grid carbon intensity. By simulating interventions such as temporal shifting, task interruption, and hardware scaling on real-world genomics and earth observation pipelines, we demonstrate that carbon footprints can be reduced by up to 66% with minimal impact on completion times.
Introduction
Scientific research increasingly relies on complex computational pipelines for data-intensive tasks. These workflows often run for extended periods on large clusters, leading to substantial energy consumption. Carbon-aware computing offers a pathway to sustainability by scheduling workloads when renewable energy is most prevalent on the grid. Scientific workflows are uniquely suited for this approach due to their delay tolerance, interruptibility, and scalability. This paper seeks to quantify the potential carbon savings achievable through intelligent scheduling and resource management.
Methodology
The study utilizes Ichnos, a tool that estimates carbon footprints by applying linear power models to historical execution traces. These traces are mapped to real-world Carbon Intensity (CI) data, considering both average and marginal emissions. The researchers evaluated two workflows from the nf-core library: Chip-Seq (genomics) and Rangeland (earth observation).
Three primary interventions were simulated:
- Entire Workflow Shifting: Delaying the start of a workflow to coincide with greener energy windows.
- Interrupted Workflow Shifting: Pausing execution between tasks to avoid carbon intensity spikes.
- Resource and Hardware Scaling: Adjusting CPU frequencies and node counts to "shape" the workload to fit within low-carbon windows.
Key Insights
A critical finding is that scientific workflows are naturally interruptible. Unlike monolithic applications, workflows save intermediate data to disk, allowing them to be paused and resumed with negligible overhead. This flexibility enables "interrupted shifting," where tasks are executed across multiple non-consecutive green windows, often yielding higher savings than simple start-time delays. Furthermore, workload "shaping" through scaling allows researchers to compress execution times to fit within brief windows of highly renewable energy without necessarily increasing total energy consumption.
Results
The baseline carbon footprint for the Chip-Seq workflow was approximately 27.6 kgCO2e, while Rangeland generated roughly 10.0 kgCO2e. Implementing a 192-hour shifting window for Chip-Seq resulted in a dramatic reduction of up to 66% in marginal CI emissions. For the Rangeland workflow, interrupted shifting achieved a 52% reduction with only a 3% increase in total runtime.
Hardware scaling also proved effective; increasing the node count for Chip-Seq from 2 to 8 reduced the carbon footprint by 42.5% by enabling the job to complete within a specific low-carbon window.
Conclusion and Future Work
This research establishes an upper bound for carbon savings in scientific computing, assuming perfect grid forecasts and task duration knowledge. Future efforts should focus on developing dynamic, real-time carbon-aware schedulers capable of handling the uncertainties of grid fluctuations and task variability. While this study focuses on operational carbon, future assessments should also incorporate the embodied carbon of the underlying hardware to provide a holistic view of computational sustainability.
Summary updated