DAMOV: A New Methodology and Benchmark Suite for Evaluating Data Movement Bottlenecks

Geraldo F. Oliveira, Juan Gómez-Luna, Lois Orosa, Saugata Ghose, Nandita Vijaykumar, Ivan Fernandez, Mohammad Sadrosadati, Onur Mutlu

TL;DR

Data movement between the CPU and memory is the single greatest hurdle for modern system performance and energy efficiency. This paper introduces DAMOV, a rigorous three-step methodology and an extensive open-source benchmark suite that classifies applications into six distinct "data movement bottleneck" classes, providing a roadmap for when to use traditional caching versus emerging Near-Data Processing (NDP) architectures.

What Problem Does This Paper Address?

For decades, we’ve lived with the "Memory Wall": processors get faster much more quickly than memory does. Today, moving data from DRAM to the CPU consumes orders of magnitude more energy than the actual computation.

While architects have proposed Near-Data Processing (NDP)—placing logic directly inside or next to memory—to solve this, we lack a systematic way to identify which applications actually benefit from it. Common metrics like the Roofline Model or MPKI (Misses Per Kilo-Instruction) are often too coarse; they might tell you an app is "memory-bound," but they don't explain why (is it latency? bandwidth? cache contention?) or if moving it to an NDP unit (which often lacks deep cache hierarchies) will actually make it slower.

Methods

The authors developed a systematic three-step workload characterization methodology:

  1. Memory-Bound Identification: Using hardware profiling (Intel VTune), they filter for functions where the "Memory Bound" metric exceeds 30%.
  2. Locality-Based Clustering: They perform architecture-independent profiling to measure inherent spatial and temporal locality. This separates the "nature" of the application from the "limitations" of a specific CPU's cache.
  3. Bottleneck Classification: They perform a scalability analysis using a custom simulator (DAMOV-SIM), varying core counts and cache sizes. They introduce a new metric, LFMR (Last-to-First Miss-Ratio)—the ratio of Last-Level Cache (LLC) misses to L1 misses.
  4. High LFMR (~1): The cache hierarchy is useless; data goes straight to DRAM.
  5. Low LFMR (~0): The cache is doing a great job; the bottleneck is likely internal to the CPU.

Datasets

The study is massive in scope:

  • Initial Characterization: 77,000 functions across 345 applications from 37 different benchmark suites (including PARSEC, Rodinia, SPEC, and graph frameworks).
  • The DAMOV Suite: A curated set of 144 functions from 74 applications that represent the most diverse data movement behaviors.
  • Simulation: Evaluation was performed using DAMOV-SIM, which integrates the ZSim CPU simulator with the Ramulator DRAM simulator.

Key Insights and Ideas

The "Aha!" moment of the paper is the discovery of six distinct classes of data movement bottlenecks:

  • Class 1a (DRAM Bandwidth-Bound): Low locality, high MPKI. These are the "golden children" for NDP, seeing massive speedups from the high internal bandwidth of 3D-stacked memory.
  • Class 1b (DRAM Latency-Bound): Low locality, low MPKI. NDP helps here not through bandwidth, but by cutting out the "hops" through the cache hierarchy and off-chip links.
  • Class 1c (L1/L2 Capacity-Bound): These apps benefit from NDP at low core counts, but as you add more CPU cores (and thus more aggregate cache), the CPU eventually wins.
  • Class 2a (L3 Contention-Bound): High temporal locality, but they "fight" over the L3 cache. NDP provides a cost-effective bypass.
  • Class 2b (L1 Capacity-Bound): These are bottlenecked so early in the pipe that NDP and CPUs perform similarly.
  • Class 2c (Compute-Bound): High arithmetic intensity. Crucially, NDP can hurt these applications because they lose the sophisticated out-of-order execution and deep caches of a traditional host CPU.

Key Results

  • NDP Potential: For Class 1a (Bandwidth-bound), NDP architectures provided up to 4.8x speedup over a traditional host CPU.
  • Energy Savings: NDP can reduce total system energy by up to 89% for certain workloads by eliminating the power-hungry off-chip interconnect traffic.
  • Classification Accuracy: The authors' methodology was able to accurately classify 97% of a "validation set" of 100 previously unseen memory-bound functions.
  • The "Cache Trap": They demonstrated that for Class 2c, the host CPU is up to 77% more energy-efficient than NDP because the CPU's caches successfully filter almost all DRAM traffic.

Limitations and Future Work

  • NDP Design Space: The study primarily uses general-purpose in-order and out-of-order cores for NDP. The authors acknowledge that specialized accelerators (like Systolic Arrays for AI) might shift these bottleneck classifications.
  • Offloading Overheads: The analysis is function-level. In a real system, the cost of "packaging" data to send to an NDP unit and maintaining cache coherence between the host and memory remains a significant "open research problem."
  • Software Mapping: While the paper identifies what to offload, it doesn't provide an automated compiler toolchain to do the offloading, which remains a hurdle for widespread NDP adoption.

Summary updated

Abstract

Data movement between the CPU and main memory is a first-order obstacle against improv ing performance, scalability, and energy efficiency in modern systems. Computer systems employ a range of techniques to reduce overheads tied to data movement, spanning from traditional mechanisms (e.g., deep multi-level cache hierarch ies, aggressive hardware prefetcher s) to emerging techniques such as Near-Data Processing (NDP), where some computation is moved close to memory. Prior NDP works investigate the root causes of data movement bottlenecks using different profiling methodologies and tools. However, there is still a lack of understanding about the key metrics that can identify different data movement bottlenecks and their relation to traditional and emerging data movement mitigation mechanisms. Our goal is to methodically identify potential sources of data movement over a broad set of applications and to comprehensively compare traditional compute-centric data movement mitigation techniques (e.g., cach ing and prefetch ing) to more memory-centric techniques (e.g., NDP), thereby developing a rigorous understanding of the best techniques to mitigate each source of data movement. With this goal in mind, we perform the first large-scale characterization of a wide variety of applications, across a wide range of application domains, to identify fundamental program properties that lead to data movement to/from main memory. We develop the first systematic methodology to classify applications based on the sources contributing to data movement bottlenecks. From our large-scale characterization of 77K functions across 345 applications, we select 144 functions to form the first open-source benchmark suite (DAMOV) for main memory data movement studies. We select a diverse range of functions that (1) represent different types of data movement bottlenecks, and (2) come from a wide range of application domains. Using NDP as a case study, we identify new insights about the different data movement bottlenecks and use these insights to determine the most suitable data movement mitigation mechanism for a particular application. We open-source DAMOV and the complete source code for our new characterization methodology at https://github.com/CMU-SAFARI/DAMOV .
BibTeX
@inproceedings{oliveira2021damov,
  author = {Geraldo F. Oliveira and Juan Gómez-Luna and Lois Orosa and Saugata Ghose and Nandita Vijaykumar and Ivan Fernandez and Mohammad Sadrosadati and Onur Mutlu},
  title = {{DAMOV: A New Methodology and Benchmark Suite for Evaluating Data Movement Bottlenecks}},
  booktitle = {IEEE Access},
  year = {2021},
  url = {https://openreview.net/forum?id=sn3g9mEDBe},
}

Loading related content...

Figure