TL;DR
This paper introduces a novel runtime architecture that seamlessly integrates traditional High-Performance Computing (HPC) with Machine Learning (ML) for scientific workflows. By extending the RADICAL-Pilot system with service-based execution, it enables scalable, distributed ML inference and training across local and remote HPC/cloud resources, paving the way for more powerful AI-driven scientific discovery.
What Problem Does This Paper Address?
Modern scientific research increasingly relies on the synergy between HPC and ML. However, effectively combining these two domains for complex, data-driven workflows presents significant challenges. Traditional approaches often involve ad-hoc scripting or specialized systems that lack flexibility, interoperability, and scalability. This paper tackles the problem of creating a unified, scalable runtime system that can orchestrate hybrid HPC/ML applications, allowing ML models to be integrated "out of" HPC applications (e.g., for steering, analysis, or hypothesis generation) across diverse and distributed computing environments. The goal is to bridge the gap between the computational power of HPC and the analytical capabilities of ML without introducing significant architectural overheads.
Methods
The core of this work is the extension of the RADICAL-Pilot runtime system. The authors introduce a service-based execution model where ML capabilities are exposed as services. This involves:
- Service Abstraction: Defining a "Service Task" that extends RADICAL-Pilot's existing Task abstraction. This allows ML models to be managed as tasks within the runtime.
- Service Manager: A new component that handles the lifecycle of these services, including launching, monitoring, and managing their availability.
- Unified API: Services expose a well-defined, general-purpose API (initially using ZeroMQ and REST) that allows clients (HPC tasks or other services) to interact with ML models without needing to know the underlying ML framework or model specifics.
- Extensible Architecture: The design is intended to be extensible, allowing for the integration of various ML serving and hosting technologies (like Ollama, VLLM, TensorRT, DeepSpeed) and supporting different types of ML tasks (e.g., inference, training).
- Data Management: Integration with services like Globus for efficient data staging and transfer, crucial for large scientific datasets.
The architecture is designed to be agnostic to the specific ML code, enabling it to support a wide range of AI-out-HPC scenarios.
Datasets
The paper describes three use cases that guide the design and evaluation:
- Cell Painting Pipeline: Uses a cell-painting microscopy dataset (~1.6 TB) for image classification and signature detection. Data augmentation is performed online.
- Signature Detection Pipeline: Analyzes DNA variants from ~300 MB VCF files for radiation-induced mutational patterns. It also plans to incorporate ~20 GB of literature data for LLM-based hypothesis generation.
- Uncertainty Quantification (UQ) Pipeline: Uses a small dataset (~3.4 MB) of plain text question-and-answer pairs for evaluating LLM uncertainty.
The experiments for performance characterization use a "NOOP" model (which returns immediately) and the "llama-8b" model hosted by Ollama. No new benchmarks were introduced; instead, the focus was on evaluating the performance of the proposed runtime architecture using these existing or planned datasets.
Key Insights and Ideas
- Service-Oriented HPC/ML Integration: The most significant idea is treating ML capabilities as first-class services within an HPC runtime. This decouples ML models from the core HPC execution, enabling flexibility and scalability.
- Abstraction is Key: The runtime's success hinges on its ability to abstract away the complexities of ML model serving and execution, providing a unified interface for HPC applications.
- Minimal Overhead: Preliminary results suggest that this service-based approach can be implemented with minimal architectural overhead, which is crucial for performance-sensitive HPC environments.
- Network Latency vs. Inference Time: For remote services, network latency is a significant factor in response time, but for actual inference tasks, the model's computation time dominates, highlighting the importance of optimizing ML model performance itself.
- Scalability is Achievable: The architecture demonstrates promising scalability for bootstrapping service instances and handling inference requests, both locally and remotely.
Key Results
- Bootstrap Time (BT): The time to launch and initialize local ML service instances was measured. For up to 160 instances, the launch time was relatively constant, with model loading and initialization being the dominant factor. The architecture scales to at least 640 concurrent service instances on a leadership-class HPC system (Frontier).
- Response Time (RT): For NOOP (no actual inference) calls, response times were dominated by network latency, especially for remote services. However, increasing the number of concurrent service instances effectively reduced the impact of latency by allowing more requests to be in flight.
- Inference Time (IT): When running actual LLM inference (llama-8b), the inference duration itself became the primary contributor to the overall response time, overshadowing network latency. This indicates that optimizing the ML models and their serving infrastructure is critical for performance.
- Scalability: The system showed good strong and weak scaling for both local and remote scenarios, demonstrating its potential for handling increasing workloads.
Limitations and Future Work
- Limited ML Serving Technologies: The current prototype primarily uses Ollama for ML model serving. The authors acknowledge that more specialized and high-performing technologies (VLLM, TensorRT, DeepSpeed) are needed for optimal performance at scale and plan to integrate them.
- Single-Threaded Services: Current service instances are single-threaded, meaning they can only handle one request at a time, leading to queuing. Future work will address this by exploring multi-threaded or parallelized model serving.
- Rudimentary Load Balancing: The current load balancing is basic. Future work will involve developing more sophisticated dynamic load balancing strategies.
- Focus on AI-out-HPC: While the architecture is designed to be general, the current prototyping and evaluation focus on AI-out-HPC scenarios.
- Future Work: The authors plan to:
- Integrate advanced ML serving and hosting technologies.
- Support distributed online model training (e.g., PyTorch FSDP).
- Explore adaptive resource scheduling for dynamic load balancing.
- Conduct more extensive experiments with larger-scale use cases and more complex ML models.
- Compare performance against HPC-only baselines.
Summary updated