Workflow Mini-Apps: Portable, Scalable, Tunable & Faithful Representations of Scientific Workflows

Ozgur O. Kilic, Tianle Wang, Matteo Turilli, М. И. Титов, André Merzky, Line Pouchard, Shantenu Jha

Abstract

Workflows are critical for scientific discovery. However, the sophistication, heterogeneity, and scale of workflows make building, testing, and optimizing them increasingly challenging. Furthermore, their complexity and heterogeneity make performance reproducibility hard. In this paper, we propose workflow mini-apps as a tool to address the challenges in building and testing workflows while controlling the fidelity of representing real-world workflows. Workflow mini-apps are deployed and run on various HPC systems and architectures without workflow-specific constraints. We offer insight into their design and implementation, providing an analysis of their performance and reproducibility. Workflow mini-apps thus advance the science of workflows by providing simple, portable, and managed (fidelity) representations of otherwise complex and difficult-to-control real workflows.

Introduction

Modern scientific discovery increasingly relies on "workflows"—complex sequences of tasks that integrate heavy-duty simulations with AI and machine learning. However, the success of these workflows has introduced significant challenges for High-Performance Computing (HPC) researchers. Workflows are often massive, highly heterogeneous, and tightly coupled to specific software libraries or hardware configurations.

This creates three primary obstacles:

  1. Portability: Migrating a workflow between different supercomputing architectures (e.g., from NVIDIA-based Summit to AMD-based Frontier) often requires extensive re-coding and dependency management.
  2. Reproducibility: System noise and inherent complexity make it difficult to distinguish between performance issues caused by the code versus the underlying hardware.
  3. Optimization Cost: Testing new scheduling algorithms or system configurations using full-scale scientific experiments is prohibitively expensive and slow.

While traditional "proxy apps" emulate individual tasks or math kernels, they fail to capture the complexity of an entire workflow, including task interdependencies and data movement.

Methodology

To address these challenges, we propose a three-step methodology for creating workflow mini-apps: lightweight, portable emulators that mimic the performance behavior of complex scientific workflows without the associated software dependencies.

The three-step process for creating a workflow mini-app, from initial analysis to final deployment.
  1. Analysis: The original workflow is profiled to identify task categories, dependencies, and resource consumption patterns (CPU, GPU, and I/O).
  2. Generation: Using the wfMiniAPI (available in Python and C++), researchers create "black box" tasks. These tasks execute tunable kernels—such as matrix multiplications or synthetic I/O—that replicate the resource footprint of the original scientific code.
  3. Assembly & Fine-tuning: These emulated tasks are linked using workflow middleware (e.g., RADICAL Cybertools). The mini-app is then tuned to ensure its execution time and resource utilization are a predictable, faithful fraction of the original.

Applications

The approach was validated using two representative workflows from the Exascale Computing Project (ECP):

  • The Inverse Problem (IP): A stationary workflow used in neutron scattering to predict material properties, representing a standard, predictable pipeline.
  • DeepDriveMD: An adaptive workflow for molecular dynamics that uses AI to steer simulations in real-time, representing a complex, dynamic structure.

Key Results

The workflow mini-apps demonstrated high fidelity, mirroring the CPU, GPU, and I/O utilization patterns of the original applications across various configurations.

Experimental results comparing the makespan and resource utilization of original applications (IP, DDMD) and their corresponding mini-apps (m-app) across different systems (V1, V2, V3).
  • Performance Reproducibility: Run-to-run variation for the mini-apps was less than 3%, matching or exceeding the stability of the original scientific codes.
  • Seamless Portability: The same mini-app was successfully deployed on three flagship supercomputers—Polaris, Summit, and Frontier—with minimal effort, despite their differing architectures.
  • Scalability Insights: Mini-apps accurately identified scaling bottlenecks, such as AI tasks that failed to scale beyond 16 nodes, providing the same strong scaling curves as the full-scale applications.

Conclusion

Workflow mini-apps provide a portable and manageable way to represent complex scientific workflows. While currently requiring manual effort for analysis and tuning, future work aims to automate these processes. Expanding the wfMiniAPI to include more diverse behaviors and support more complex Directed Acyclic Graph (DAG) structures will further enhance the utility of these tools for the HPC community.

Summary updated

Abstract

Workflows are critical for scientific discovery. However, the sophistication, heterogeneity, and scale of workflows make building, testing, and optimizing them increasingly challenging. Furthermore, their complexity and heterogeneity make performance reproducibility hard. In this paper, we propose workflow mini-apps as a tool to address the challenges in building and testing workflows while controlling the fidelity of representing real-world workflows. Workflow mini-apps are deployed and run on various HPC systems and architectures without workflow-specific constraints. We offer insight into their design and implementation, providing an analysis of their performance and reproducibility. Workflow mini-apps thus advance the science of workflows by providing simple, portable, and managed (fidelity) representations of otherwise complex and difficult-to-control real workflows.
BibTeX
@misc{kilic2024workflow,
  author = {Ozgur O. Kilic and Tianle Wang and Matteo Turilli and М. И. Титов and André Merzky and Line Pouchard and Shantenu Jha},
  title = {{Workflow Mini-Apps: Portable, Scalable, Tunable \& Faithful Representations of Scientific Workflows}},
  year = {2024},
  eprint = {2403.18073},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2403.18073},
}
Figure