DAMOV: A New Methodology and Benchmark Suite for Evaluating Data Movement Bottlenecks

Geraldo F. Oliveira, Juan Gómez-Luna, Lois Orosa, Saugata Ghose, Nandita Vijaykumar, Ivan Fernandez, Mohammad Sadrosadati, Onur Mutlu

Abstract

Data movement between the CPU and main memory is a first-order obstacle against improving performance, scalability, and energy efficiency in modern systems. Computer systems employ a range of techniques to reduce overheads tied to data movement, spanning from traditional mechanisms (e.g., deep multi-level cache hierarchies, aggressive hardware prefetchers) to emerging techniques such as Near-Data Processing (NDP), where some computation is moved close to memory. Our goal is to methodically identify potential sources of data movement over a broad set of applications and to comprehensively compare traditional compute-centric data movement mitigation techniques to more memory-centric techniques, thereby developing a rigorous understanding of the best techniques to mitigate each source of data movement. With this goal in mind, we perform the first large-scale characterization of a wide variety of applications, across a wide range of application domains, to identify fundamental program properties that lead to data movement to/from main memory. We develop the first systematic methodology to classify applications based on the sources contributing to data movement bottlenecks. From our large-scale characterization of 77K functions across 345 applications, we select 144 functions to form the first open-source benchmark suite (DAMOV) for main memory data movement studies. We select a diverse range of functions that (1) represent different types of data movement bottlenecks, and (2) come from a wide range of application domains. Using NDP as a case study, we identify new insights about the different data movement bottlenecks and use these insights to determine the most suitable data movement mitigation mechanism for a particular application.

Introduction: The Data Movement Challenge

For decades, computer architecture has been constrained by the "Memory Wall"—the widening gap between processor speed and memory latency. In modern systems, moving data from DRAM to the CPU can consume orders of magnitude more energy than the actual computation. While Near-Data Processing (NDP) has emerged as a promising solution by placing logic closer to memory, architects lack a systematic way to determine which applications truly benefit from it. Traditional metrics like the Roofline Model or Misses Per Kilo-Instruction (MPKI) are often too coarse to distinguish between different types of memory bottlenecks, such as latency, bandwidth, or cache contention.

The DAMOV Methodology

To address this, the authors propose a systematic three-step methodology for workload characterization that decouples application behavior from specific hardware limitations.

The DAMOV three-step methodology for identifying and classifying data movement bottlenecks.
  1. Memory-Bound Identification: Using hardware profiling (e.g., Intel VTune) to identify functions where the "Memory Bound" metric exceeds 30%.
  2. Locality-Based Clustering: Performing architecture-independent profiling to measure inherent spatial and temporal locality. This captures the fundamental "nature" of the application's data access patterns.
  3. Bottleneck Classification: Conducting scalability analysis using a custom simulator (DAMOV-SIM). The methodology introduces the Last-to-First Miss-Ratio (LFMR)—the ratio of Last-Level Cache (LLC) misses to L1 misses—to determine how effectively the cache hierarchy filters memory traffic.

Data Movement Bottleneck Classes

Large-scale characterization of 77,000 functions led to the discovery of six distinct classes of data movement bottlenecks. These classes provide a roadmap for selecting the most effective mitigation strategy.

Mapping of application functions into six distinct bottleneck classes based on locality metrics.
  • Class 1a (DRAM Bandwidth-Bound): Characterized by low locality and high MPKI. These workloads see the greatest benefit from NDP due to the high internal bandwidth of 3D-stacked memory.
  • Class 1b (DRAM Latency-Bound): Low locality and low MPKI. NDP helps by reducing the latency of traversing the cache hierarchy and off-chip links.
  • Class 1c (L1/L2 Capacity-Bound): These benefit from NDP at low core counts, but traditional CPUs can eventually outperform them as aggregate cache size increases with more cores.
  • Class 2a (L3 Contention-Bound): High temporal locality but high L3 contention. NDP offers a way to bypass the congested cache hierarchy.
  • Class 2b (L1 Capacity-Bound): Bottlenecked very early in the pipeline; NDP and CPUs show similar performance.
  • Class 2c (Compute-Bound): High arithmetic intensity. These applications are often faster on traditional CPUs, which feature sophisticated out-of-order execution and deep caches that NDP units typically lack.

Key Results and Insights

Using the DAMOV benchmark suite—a curated set of 144 functions representing these diverse behaviors—the study evaluated the potential of NDP:

  • Performance Gains: Class 1a workloads achieved up to a 4.8x speedup on NDP architectures compared to traditional host CPUs.
  • Energy Efficiency: NDP can reduce total system energy by up to 89% by eliminating power-hungry off-chip interconnect traffic.
  • Classification Accuracy: The methodology accurately classified 97% of a validation set of 100 previously unseen memory-bound functions.
  • The "Cache Trap": For Class 2c (Compute-Bound) workloads, the host CPU is up to 77% more energy-efficient because its caches successfully filter nearly all DRAM traffic, a benefit lost when offloading to simpler NDP units.

Conclusion and Future Directions

DAMOV provides the first open-source benchmark suite and methodology specifically designed for studying data movement bottlenecks. While it offers a clear path for identifying offloading candidates, several challenges remain. Future work includes exploring the NDP design space for specialized accelerators, addressing the overheads of data packaging and cache coherence, and developing automated compiler toolchains to facilitate seamless offloading.

Summary updated

Abstract

Data movement between the CPU and main memory is a first-order obstacle against improving performance, scalability, and energy efficiency in modern systems. Computer systems employ a range of techniques to reduce overheads tied to data movement, spanning from traditional mechanisms (e.g., deep multi-level cache hierarchies, aggressive hardware prefetchers) to emerging techniques such as Near-Data Processing (NDP), where some computation is moved close to memory. Our goal is to methodically identify potential sources of data movement over a broad set of applications and to comprehensively compare traditional compute-centric data movement mitigation techniques to more memory-centric techniques, thereby developing a rigorous understanding of the best techniques to mitigate each source of data movement. With this goal in mind, we perform the first large-scale characterization of a wide variety of applications, across a wide range of application domains, to identify fundamental program properties that lead to data movement to/from main memory. We develop the first systematic methodology to classify applications based on the sources contributing to data movement bottlenecks. From our large-scale characterization of 77K functions across 345 applications, we select 144 functions to form the first open-source benchmark suite (DAMOV) for main memory data movement studies. We select a diverse range of functions that (1) represent different types of data movement bottlenecks, and (2) come from a wide range of application domains. Using NDP as a case study, we identify new insights about the different data movement bottlenecks and use these insights to determine the most suitable data movement mitigation mechanism for a particular application. We open-source DAMOV and the complete source code for our new characterization methodology at https://github.com/CMU-SAFARI/DAMOV.
BibTeX
@inproceedings{oliveira2021damov,
  author = {Geraldo F. Oliveira and Juan Gómez-Luna and Lois Orosa and Saugata Ghose and Nandita Vijaykumar and Ivan Fernandez and Mohammad Sadrosadati and Onur Mutlu},
  title = {{DAMOV: A New Methodology and Benchmark Suite for Evaluating Data Movement Bottlenecks}},
  booktitle = {IEEE Access},
  year = {2021},
  url = {https://openreview.net/forum?id=sn3g9mEDBe},
}

Loading related content...

Figure