HETOCompiler: An MLIR-based crypTOgraphic Compilation Framework for HEterogeneous Devices

Zhiyuan Tan, Liutong Han, Mingjie Xing, Yanjun Wu

Abstract

In the era of diminishing returns from Moore’s Law, heterogeneous computing systems have emerged as a vital approach to enhance computational efficiency. This paper introduces a novel MLIR-based dialect, named hyper, designed to optimize data management and parallel computation across diverse hardware architectures. The hyper dialect abstracts the complexities of heterogeneous computing by providing a unified compilation framework that efficiently schedules tasks and manages data communication. To demonstrate its capabilities, we present HETOCompiler, a cryptography-focused compiler prototype that implements multiple hash algorithms and enables their execution on heterogeneous systems. The proposed approach achieves performance improvements over existing programming models for heterogeneous computing (OpenCL), offering an average speedup of 1.93x, 1.18x, and 1.12x for SHA-1, MD5, and SM3 algorithms, respectively. Our findings highlight the potential of the hyper dialect in harnessing the full computational power of heterogeneous devices, advancing the field of compiler design for heterogeneous systems.

Introduction

As the physical limits of single-core processor scaling are reached, the industry has shifted toward heterogeneous computing—leveraging the combined power of CPUs, GPUs, and specialized accelerators. However, programming these systems remains a significant challenge. Developers are often forced to write device-specific code (e.g., CUDA for GPUs and C++ for CPUs) and manually orchestrate complex data transfers between memory spaces.

While the Multi-Level Intermediate Representation (MLIR) ecosystem offers a robust foundation for modern compiler construction, it lacks a unified abstraction for describing how tasks should be partitioned and scheduled across disparate hardware types simultaneously. HETOCompiler addresses this gap by providing a high-level abstraction that enables a single source representation to execute efficiently across hybrid systems.

The HETOCompiler Framework

HETOCompiler is built upon the MLIR ecosystem and introduces two primary customized dialects to manage the complexity of heterogeneous cryptographic workloads:

  1. The hyper Dialect: Acting as the framework's core, this dialect introduces operations such as hyper.for for parallel loops that can be split across devices and hyper.memcpy for abstracting data movement. It allows the compiler to determine workload distribution—for instance, assigning 20% of a task to the CPU and 80% to the GPU based on target configurations.
  2. The crypto Dialect: A domain-specific layer that describes cryptographic operations (SHA-1, MD5, SM3) at a high level, enabling optimizations before the code is lowered to machine-specific instructions.
Figure 1: The HETOCompiler compilation pipeline, showing the transition from customized dialects to official MLIR dialects and finally to hardware backends.

Key Innovations in Task Scheduling and Optimization

The framework employs several advanced techniques to bridge the gap between high-level code and heterogeneous hardware:

Automatic Lowering and Task Splitting

The framework automatically translates high-level hyper operations into device-specific instructions. As shown in the figure below, a single hyper.for loop containing a cryptographic operation can be split into an async.execute block for the GPU (using gpu.launch) and another for the CPU (using scf.parallel). This allows for simultaneous execution across different hardware backends without manual developer intervention.

Figure 2: Illustration of task splitting where a single hyper.for loop is lowered into GPU-specific and CPU-specific operations for parallel execution.

Graph-Based Memory Optimization

To minimize the overhead of data transfers, the compiler constructs a directed graph of memory allocations. An optimization pass identifies and eliminates redundant host-to-device copies if the required data is already present on the target device, significantly reducing communication latency.

Hardware-Awareness

HETOCompiler includes a "Hardware Detector" that identifies specific CPU features, such as Intel's SHA-NI (Secure Hash Algorithm New Instructions). By detecting these features at compile time, the framework can select the most efficient implementation for the specific silicon available.

Experimental Evaluation

The researchers evaluated HETOCompiler using a massive-scale hashing scenario, processing billions of short messages (totaling approximately 27 GB for SHA-1, 23 GB for MD5, and 38 GB for SM3). Testing was conducted on two distinct server configurations: one featuring a consumer-grade i7 with SHA-NI support and an RTX 2060 Super, and a second featuring a high-end Xeon Gold (without SHA-NI) and an RTX 3090.

Performance Results

HETOCompiler was compared against OpenCL (the industry standard for cross-platform parallel programming) and OpenSSL (a highly optimized CPU-centric library):

  • Vs. OpenCL: HETOCompiler achieved average speedups of 1.93x for SHA-1, 1.18x for MD5, and 1.12x for SM3.
  • Vs. OpenSSL: In massive parallel hashing scenarios, the compiler-generated code was up to 55.8x faster for SHA-1, primarily due to its ability to effectively offload workloads to the GPU.
  • Workload Distribution: The study confirmed that the optimal "sweet spot" for task splitting varies by hardware and algorithm. For example, the presence of SHA-NI on the i7 allowed the CPU to contribute more effectively to the total throughput compared to the Xeon processor.

Conclusion and Future Work

HETOCompiler demonstrates that a dialect-based approach in MLIR can effectively hide the complexities of heterogeneous programming while delivering superior performance. By treating CPUs and GPUs as a unified computational resource rather than isolated entities, the framework achieves significant gains in cryptographic throughput.

Future work will focus on expanding the hyper dialect to support more complex topologies, such as multi-GPU clusters and FPGAs. Additionally, the researchers aim to replace manual workload ratio selection with Machine Learning-based scheduling to automatically predict the optimal task split for any given hardware configuration.

Summary updated

Abstract

In the era of diminishing returns from Moores Law, heterogeneous computing systems have emerged as a vital approach to enhance computational efficiency. This paper introduces a novel MLIR-based dialect, named hyper, designed to optimize data management and parallel computation across diverse hardware architectures. The hyper dialect abstracts the complexities of heterogeneous computing by providing a unified compilation framework that efficiently schedules tasks and manages data communication. To demonstrate its capabilities, we present HETOCompiler, a cryptography-focused compiler prototype that implements multiple hash algorithms and enables their execution on heterogeneous systems. The proposed approach achieves performance improvements over existing programming models for heterogeneous computing (OpenCL), offering an average speedup of 1.93x, 1.18x, and 1.12x for SHA-1, MD5, and SM3 algorithms, respectively. Our findings highlight the potential of the hyper dialect in harnessing the full computational power of heterogeneous devices, advancing the field of compiler design for heterogeneous systems.
BibTeX
@misc{tan2024hetocompiler,
  author = {Zhiyuan Tan and Liutong Han and Mingjie Xing and Yanjun Wu},
  title = {{HETOCompiler: An MLIR-based crypTOgraphic Compilation Framework for HEterogeneous Devices}},
  year = {2024},
  eprint = {2407.09333},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2407.09333},
}
Figure