Abstract
The rapidly growing popularity and scale of data-parallel workloads demand a corresponding increase in raw computational power of GPUs. As single-GPU systems struggle to satisfy performance demands, multi-GPU systems have begun to dominate high-performance computing. However, the research community lacks a comprehensive, publicly available simulation framework to evaluate these designs. We present MGSim, a cycle-accurate, multi-threaded simulator based on AMD's GCN3 ISA, and MGMark, a suite of multi-GPU workloads. MGSim achieves a 5.5% average performance error compared to real hardware and significant simulation speedups. We demonstrate its utility through a case study comparing Unified (U-MGPU) and Discrete (D-MGPU) programming models.
Introduction
As data-parallel workloads like Deep Learning and scientific simulations grow in scale, single GPUs are hitting physical limits on die size and power. While the industry is moving toward multi-GPU systems, these architectures face significant hurdles: CPU-to-GPU synchronization, limited interconnect bandwidth, and complex memory management. Research has been hindered by a lack of tools; existing simulators are often restricted to single-GPU models or are too slow for large-scale systems. MGSim addresses this gap by providing a cycle-accurate, modular, and scalable framework.
MGSim: A High-Fidelity Multi-GPU Simulator
MGSim is built using the Go programming language to leverage native concurrency. Unlike simulators that use high-level abstractions, MGSim models actual data flow and timing for every message and instruction, ensuring "no magic" in its results.
To address the inherent slowness of architectural simulation, MGSim employs a conservative parallel simulation scheme. This multi-threaded engine ensures that events scheduled at the same time-step are processed in parallel without compromising chronological accuracy. Furthermore, its modular, component-based architecture allows researchers to easily plug in new interconnect models or cache policies. MGSim executes unmodified AMD Graphics Core Next 3 (GCN3) binaries, making it highly relevant to modern hardware research.
MGMark: Benchmarking Collaborative Patterns
We complement MGSim with MGMark, a benchmark suite designed to categorize and stress multi-GPU collaborative execution patterns. The suite includes workloads covering various data access patterns:
- Partitioned Data: AES, KMeans.
- Adjacent Access: FIR, Simple Convolution (SC).
- Gather/Scatter: Gradient Descent (GD), Matrix Transpose (MT).
- Irregular: Bitonic Sort (BS).
Validation against an AMD R9 Nano GPU shows that MGSim's performance results are within a 5.5% average error of actual hardware.
Case Study: U-MGPU vs. D-MGPU
A key capability of MGSim is exploring different programming models. We compared a Unified multi-GPU system (U-MGPU), where the system appears as one large logical GPU, against a Discrete multi-GPU system (D-MGPU), where the programmer explicitly manages multiple devices.
Our evaluation revealed a "Locality Paradox." While U-MGPU is easier to program, it often performs significantly worse because it hides data locality, leading to excessive cross-GPU traffic. In contrast, D-MGPU allows for better data-affinity scheduling. The results show a near-perfect correlation between cross-GPU traffic and execution slowdown, highlighting the interconnect as the ultimate bottleneck.
Conclusion
MGSim and MGMark provide a robust foundation for multi-GPU research. MGSim achieves approximately 27 KIPS, significantly faster than older state-of-the-art simulators, and scales well with multiple CPU cores. Using 4 cores, the simulator achieved a 3.5x speedup in function emulation and a 2.5x speedup in detailed architectural simulation. Future work will focus on evolving the ISA support (e.g., to GCN5/Vega), improving system-level atomic operations, and exploring more diverse network topologies for larger 8 or 16 GPU configurations.
Summary updated