TL;DR
This paper introduces "hyper," a new MLIR dialect designed to simplify and optimize parallel computation across diverse heterogeneous hardware (CPUs, GPUs, etc.). Their HETOCompiler prototype, built using this dialect, demonstrates significant performance gains for cryptographic hash functions compared to existing methods like OpenCL, by intelligently managing data and task scheduling.
What Problem Does This Paper Address?
As Moore's Law slows, heterogeneous computing (combining different types of processors like CPUs and GPUs) is crucial for boosting computational efficiency. However, effectively programming these systems is complex. Compilers need to manage data movement and split tasks across various hardware backends. While MLIR offers a flexible intermediate representation, it lacks a unified way to abstract the intricacies of heterogeneous scheduling and data management. This gap makes it challenging to develop efficient compilers for these systems. This paper tackles this by proposing a new MLIR dialect, "hyper," specifically for this purpose, using cryptography as a case study.
Methods
The core of this work is the "hyper" dialect within the MLIR framework. This dialect provides high-level abstractions for:
- Data Management: Operations like
hyper.allocandhyper.deallocmanage memory allocation and deallocation on specific devices, whilehyper.memcpyhandles data transfers between host and devices. - Parallel Computation: The
hyper.foroperation defines loop structures that can be automatically partitioned and scheduled across multiple devices. It also includeshyper.reducefor efficient reduction operations.
To demonstrate its utility, the authors developed HETOCompiler, an end-to-end compiler prototype. Its workflow involves:
- Frontend: Identifies C/C++ interfaces and detects hardware information.
- Middle-Section: Utilizes the "hyper" dialect and other MLIR dialects (like "crypto") to represent computations. It then applies transformations to lower these to LLVM IR.
- Backend: Leverages LLVM's existing toolchains to generate hardware-specific machine code for CPUs and GPUs.
A key optimization technique is a graph-based host memory optimization that analyzes data management operations to reduce redundant memory overhead by mapping device memory to host memory where possible.
Datasets
Not discussed in the paper. The authors used synthetic data for their experiments, generating hash values for messages of a fixed length (9 characters) using a character set (0-9). The total data processed was substantial (e.g., 27.01 GB for SHA-1).
Key Insights and Ideas
- Unified Abstraction is Key: The "hyper" dialect effectively bridges the gap between high-level programming intent and low-level hardware specifics, abstracting away complex data movement and task scheduling.
- Cryptography as a Stress Test: Using cryptographic hash functions, which often involve massive parallel computations on short messages, proved to be an excellent use case to highlight the limitations of traditional libraries and the benefits of heterogeneous computing.
- Performance Gains Beyond OpenCL: The "hyper" dialect and HETOCompiler not only simplify development but also achieve tangible performance improvements over OpenCL, suggesting that intermediate representation-level optimization can be more effective than source-level parallel programming models.
- Hardware-Aware Optimization: The system's ability to detect hardware capabilities (like SHA-NI extensions) and adapt scheduling (e.g., workload ratios) is crucial for maximizing performance.
- I/O and Synchronization Overhead: The observed gap between theoretical optimal performance and actual measured performance points to the persistent challenge of I/O and synchronization overheads in heterogeneous systems, even with advanced compilers.
Key Results
- Speedup over OpenCL: HETOCompiler achieved an average speedup of 1.93x for SHA-1, 1.18x for MD5, and 1.12x for SM3 compared to hand-optimized OpenCL kernels.
- Significant Improvement over OpenSSL: For their specific scenario (many short messages), the compiler showed substantial improvements over OpenSSL, with average performance gains of 55.8x for SHA-1, 1.2x for MD5, and 22.5x for SM3.
- SHA-NI Impact: Server A (with SHA-NI support) showed significantly faster SHA-1 execution than Server B (without SHA-NI), demonstrating the compiler's ability to leverage hardware acceleration.
Limitations and Future Work
- Limited System Complexity: The current analysis and optimization models are primarily focused on simple heterogeneous systems (one CPU, one GPU). The authors acknowledge that these models may not directly apply to more complex configurations with multiple or varied devices.
- External Scheduling Algorithms: While the "hyper" dialect supports external workload scheduling algorithms, the investigation into these algorithms is beyond the scope of this paper.
- Future Work:
- Extend compatibility to more platforms and hardware types.
- Develop more comprehensive scheduling models for complex heterogeneous systems.
- Explore ML-based workload scheduling algorithms for advanced parallel computing scenarios.
- Further reduce I/O and synchronization overheads.
Summary updated