MGSim + MGMark: A Framework for Multi-GPU System Research

Yifan Sun, Trinayan Baruah, Saiful A. Mojumder, Shi Dong, Rafael Ubal, Xiang Gong, Shane Treadway, Yuhui Bao, Vincent Zhao, José Luis Abellán, John Kim, Ajay Joshi, David Kaeli

Abstract

The rapidly growing popularity and scale of data-parallel workloads demand a corresponding increase in raw computational power of GPUs. As single-GPU systems struggle to satisfy performance demands, multi-GPU systems have begun to dominate high-performance computing. However, the research community lacks a comprehensive, publicly available simulation framework to evaluate these designs. We present MGSim, a cycle-accurate, multi-threaded simulator based on AMD's GCN3 ISA, and MGMark, a suite of multi-GPU workloads. MGSim achieves a 5.5% average performance error compared to real hardware and significant simulation speedups. We demonstrate its utility through a case study comparing Unified (U-MGPU) and Discrete (D-MGPU) programming models.

Introduction

As data-parallel workloads like Deep Learning and scientific simulations grow in scale, single GPUs are hitting physical limits on die size and power. While the industry is moving toward multi-GPU systems, these architectures face significant hurdles: CPU-to-GPU synchronization, limited interconnect bandwidth, and complex memory management. Research has been hindered by a lack of tools; existing simulators are often restricted to single-GPU models or are too slow for large-scale systems. MGSim addresses this gap by providing a cycle-accurate, modular, and scalable framework.

MGSim: A High-Fidelity Multi-GPU Simulator

MGSim is built using the Go programming language to leverage native concurrency. Unlike simulators that use high-level abstractions, MGSim models actual data flow and timing for every message and instruction, ensuring "no magic" in its results.

To address the inherent slowness of architectural simulation, MGSim employs a conservative parallel simulation scheme. This multi-threaded engine ensures that events scheduled at the same time-step are processed in parallel without compromising chronological accuracy. Furthermore, its modular, component-based architecture allows researchers to easily plug in new interconnect models or cache policies. MGSim executes unmodified AMD Graphics Core Next 3 (GCN3) binaries, making it highly relevant to modern hardware research.

MGMark: Benchmarking Collaborative Patterns

We complement MGSim with MGMark, a benchmark suite designed to categorize and stress multi-GPU collaborative execution patterns. The suite includes workloads covering various data access patterns:

  • Partitioned Data: AES, KMeans.
  • Adjacent Access: FIR, Simple Convolution (SC).
  • Gather/Scatter: Gradient Descent (GD), Matrix Transpose (MT).
  • Irregular: Bitonic Sort (BS).

Validation against an AMD R9 Nano GPU shows that MGSim's performance results are within a 5.5% average error of actual hardware.

Case Study: U-MGPU vs. D-MGPU

A key capability of MGSim is exploring different programming models. We compared a Unified multi-GPU system (U-MGPU), where the system appears as one large logical GPU, against a Discrete multi-GPU system (D-MGPU), where the programmer explicitly manages multiple devices.

Architectural diagram of a Unified multi-GPU system (U-MGPU), where command processors are disabled on secondary GPUs to present a single logical device.
Architectural diagram of a Discrete multi-GPU system (D-MGPU), featuring active command processors across all GPUs for explicit management.

Our evaluation revealed a "Locality Paradox." While U-MGPU is easier to program, it often performs significantly worse because it hides data locality, leading to excessive cross-GPU traffic. In contrast, D-MGPU allows for better data-affinity scheduling. The results show a near-perfect correlation between cross-GPU traffic and execution slowdown, highlighting the interconnect as the ultimate bottleneck.

Conclusion

MGSim and MGMark provide a robust foundation for multi-GPU research. MGSim achieves approximately 27 KIPS, significantly faster than older state-of-the-art simulators, and scales well with multiple CPU cores. Using 4 cores, the simulator achieved a 3.5x speedup in function emulation and a 2.5x speedup in detailed architectural simulation. Future work will focus on evolving the ISA support (e.g., to GCN5/Vega), improving system-level atomic operations, and exploring more diverse network topologies for larger 8 or 16 GPU configurations.

Summary updated

Abstract

The rapidly growing popularity and scale of data-parallel workloads demand a corresponding increase in raw computational power of GPUs (Graphics Processing Units). As single-GPU systems struggle to satisfy the performance demands, multi-GPU systems have begun to dominate the high-performance computing world. The advent of such systems raises a number of design challenges, including the GPU microarchitecture, multi-GPU interconnect fabrics, runtime libraries and associated programming models. The research community currently lacks a publically available and comprehensive multi-GPU simulation framework and benchmark suite to evaluate multi-GPU system design solutions. In this work, we present MGSim, a cycle-accurate, extensively validated, multi-GPU simulator, based on AMD's Graphics Core Next 3 (GCN3) instruction set architecture. We complement MGSim with MGMark, a suite of multi-GPU workloads that explores multi-GPU collaborative execution patterns. Our simulator is scalable and comes with in-built support for multi-threaded execution to enable fast and efficient simulations. In terms of performance accuracy, MGSim differs $5.5\%$ on average when compared against actual GPU hardware. We also achieve a $3.5\times$ and a $2.5\times$ average speedup in function emulation and architectural simulation with 4 CPU cores, while delivering the same accuracy as the serial simulation. We illustrate the novel simulation capabilities provided by our simulator through a case study exploring programming models based on a unified multi-GPU system (U-MGPU) and a discrete multi-GPU system (D-MGPU) that both utilize unified memory space and cross-GPU memory access. We evaluate the design implications from our case study, suggesting that D-MGPU is an attractive programming model for future multi-GPU systems.
BibTeX
@misc{sun2018mgsim,
  author = {Yifan Sun and Trinayan Baruah and Saiful A. Mojumder and Shi Dong and Rafael Ubal and Xiang Gong and Shane Treadway and Yuhui Bao and Vincent Zhao and José Luis Abellán and John Kim and Ajay Joshi and David Kaeli},
  title = {{MGSim + MGMark: A Framework for Multi-GPU System Research}},
  year = {2018},
  eprint = {1811.02884},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1811.02884},
}
Figure