Abstract
Workflows are critical for scientific discovery. However, the sophistication, heterogeneity, and scale of workflows make building, testing, and optimizing them increasingly challenging. Furthermore, their complexity and heterogeneity make performance reproducibility hard. In this paper, we propose workflow mini-apps as a tool to address the challenges in building and testing workflows while controlling the fidelity of representing real-world workflows. Workflow mini-apps are deployed and run on various HPC systems and architectures without workflow-specific constraints. We offer insight into their design and implementation, providing an analysis of their performance and reproducibility. Workflow mini-apps thus advance the science of workflows by providing simple, portable, and managed (fidelity) representations of otherwise complex and difficult-to-control real workflows.
Introduction
Modern scientific discovery relies on massive workflows—interconnected tasks that run across High-Performance Computing (HPC) clusters. However, these workflows have become "too big to fail" and too complex to easily test. They are often bogged down by specific library dependencies, specialized hardware requirements, and massive data footprints. This complexity creates a "reproducibility crisis" in performance: moving a workflow between supercomputers makes it difficult to determine if a slowdown is due to hardware, middleware, or the code itself. Traditional "mini-apps" focus on single algorithms, failing to capture the complex inter-dependencies and resource competition inherent in full-scale workflows.
Methodology
The authors propose a three-stage methodology to create a faithful representation of a real-world workflow, centered around a new open-source library called wfMiniAPI.
- Analysis: The original workflow is deconstructed to identify task types, execution models (dependencies), and performance metrics such as CPU/GPU utilization, I/O, and total runtime (makespan).
- Generation: Using wfMiniAPI, researchers create emulated tasks. Instead of performing actual scientific computation, these tasks run "kernels" (e.g., matrix multiplication or data copies) that stress the system identically to the original code.
- Creation & Fine-tuning: These emulated tasks are assembled using workflow middleware (like RADICAL Cybertools). Parameters are fine-tuned to ensure the mini-app maintains a consistent performance ratio compared to the original, but at a fraction of the cost and complexity.
The core innovation, wfMiniAPI, provides a tunable set of computation and I/O kernels in Python and C++, enabling emulators that are portable across diverse CPU and GPU architectures (NVIDIA, AMD, etc.).
Experimental Evaluation
The approach was validated using two distinct, real-world scientific workflows across three major Department of Energy (DOE) leadership computing facilities: Polaris (ALCF), Summit (OLCF), and Frontier (OLCF).
- The Inverse Problem: A stationary workflow from the ExaLearn project used to predict material symmetry from neutron diffraction data.
- DeepDriveMD: An adaptive AI-steered molecular dynamics workflow that uses machine learning to dynamically select subsequent simulations.
Results and Key Insights
The study demonstrated that fidelity can be achieved without full functionality. By matching resource usage patterns, a mini-app acts as a reliable "stunt double" for the real workflow.
- Performance Matching: For the Inverse Problem, the mini-app's makespan and I/O ratios remained stable (varying by <1%) across different input sizes. The mini-apps exhibited run-to-run variation of <3%, comparable to the 1.2% variation seen in original workflows.
- Portability: Porting the full DeepDriveMD workflow to the Frontier supercomputer (AMD GPUs) originally required two weeks of engineering. The dependency-light workflow mini-app was ported in a fraction of that time, providing immediate performance insights.
- Scalability and Optimization: Mini-apps successfully mimicked "strong scaling" behavior. Researchers used them to test an asynchronous execution model, discovering a 10% performance boost that was later applied to the real scientific code.
Conclusion
While workflow mini-apps provide a powerful tool for benchmarking and optimization, creating them currently requires deep manual analysis of the original workflow. They also cannot predict performance changes driven by scientific conditionals (e.g., early termination based on physical states). Future work will focus on automating task emulation through execution log analysis and expanding the kernel library to mimic a wider range of scientific bottlenecks.
Summary updated