Exploring Memory Tiering Systems in the CXL Era via FPGA-based Emulation and Device-Side Management

Yiqi Chen, Xiping Dong, Zhe Zhou, Zhao Wang, Jie Zhang, Guangyu SunPact 2025 · DOI

TL;DR

This paper introduces HeteroBox, an FPGA-based emulation platform for CXL-extended heterogeneous memory systems, and HeteroMem, a novel device-side memory tiering solution. HeteroMem achieves significant performance improvements by accurately profiling data access patterns and efficiently migrating data between memory tiers, all while remaining transparent to the CPU.

What Problem Does This Paper Address?

The advent of Compute Express Link (CXL) technology promises to revolutionize memory systems by enabling the integration of diverse memory technologies (like DRAM, persistent memory, etc.) into a unified, byte-addressable memory space. This heterogeneity offers the potential for increased capacity and lower costs, but it also introduces significant challenges in performance management. Specifically, different memory types and complex CXL topologies (involving switches) lead to varying latencies and bandwidths for CPU access. Effectively managing these tiered memory systems to ensure optimal performance is crucial.

Existing solutions face several limitations:

  1. Lack of Realistic Evaluation Platforms: Current evaluation methods, such as NUMA emulation or full-system simulations (like GEM5), struggle to accurately model hardware-based tiered memory management and handle large-scale, real-world workloads. Testing with actual CXL devices is often limited by fixed configurations and the unavailability of CXL switches.
  2. Inefficiency of Software-Managed Tiering: Software-based memory tiering systems, which rely on techniques like PTE scanning or PMU sampling, often suffer from high overhead and low resolution in identifying "hot" (frequently accessed) and "cold" (infrequently accessed) data. This leads to inaccurate data placement and significant performance penalties due to software overhead.
  3. Inflexibility of Hardware-Managed Tiering: Previous hardware-based solutions typically required intrusive modifications to the CPU, making them incompatible with the dynamic and diverse configurations of CXL-extended memory systems.

Methods

The paper proposes two key innovations:

  • HeteroBox: An FPGA-based Emulation Platform:
    • Leverages real CXL-enabled FPGAs to emulate various CXL memory architectures.
  • Allows users to configure a memory space with multiple regions, each with distinct CPU-access latency and bandwidth characteristics.
  • Provides a high-fidelity platform for evaluating both software and hardware-managed memory tiering systems under realistic CXL conditions.
  • The emulation logic resides on the FPGA, intercepting memory requests and applying configurable latency and bandwidth attributes to different memory regions.
  • HeteroMem: A Device-Side Hardware-Managed Memory Tiering System:
    • Operates entirely on the device side, creating an abstraction layer between the CPU and the memory controller.
  • Remapping Unit: Translates host CPU addresses (hPA) to device physical addresses (dPA), ensuring transparency to the CPU. It also manages a remapping table to track data locations.
  • Profiling Unit: Accurately and efficiently identifies hot and cold data pages using hardware logic. It employs a Count-Min Sketch for hotness profiling in fast memory and a ping-pong bitmap for coldness profiling in fast memory.
  • Migration Unit: Handles the actual data movement between memory tiers. It performs swap operations to move hot data to faster tiers and cold data to slower tiers, achieving high bandwidth and low overhead.
  • This device-side approach makes HeteroMem transparent to the CPU and adaptable to various CXL configurations.

Datasets

Not discussed in the paper. The evaluation uses standard benchmarks.

Key Insights and Ideas

  • The Need for Device-Side Intelligence: The paper strongly advocates for moving memory management logic (tiering, profiling, migration) to the device side rather than relying on the CPU. This is crucial for achieving transparency, reducing CPU overhead, and adapting to the complexities of CXL.
  • FPGA as a Powerful Emulation Tool: HeteroBox demonstrates the effectiveness of using FPGAs to create realistic and configurable emulation environments for emerging hardware technologies like CXL, overcoming the limitations of software simulations and fixed hardware setups.
  • Hardware-Based Profiling is Key: The success of HeteroMem hinges on its accurate and low-overhead hardware-based profiling of data access patterns, which significantly outperforms software-based methods.
  • Transparency is Paramount: HeteroMem's design ensures that the CPU is unaware of the underlying memory tiering and migration, simplifying system integration and allowing existing applications to benefit without modification.
  • Migration Overhead Matters: The paper highlights that while CPU-based data migration incurs significant overhead, device-side hardware migration can achieve much higher bandwidth and lower latency.

Key Results

  • Performance Improvement: HeteroMem achieves a performance improvement of 5.1% to 16.2% over existing memory tiering solutions across various benchmarks and configurations.
  • Benchmark Performance: For the BC benchmark, HeteroMem showed up to 32.7% higher performance than NeoMem and PTE-Scan, and 44.6% higher than PEBS.
  • Migration Efficiency: Device-side hardware migration achieved 12.9x higher bandwidth compared to CPU-based migration.
  • Latency Sensitivity: HeteroMem's performance degrades minimally even with increased latency in the slow memory tier, demonstrating its robustness.
  • Reduced Memory Accesses: HeteroMem shows fewer total memory accesses compared to software-based solutions due to the elimination of CPU-based data migration overhead.

Limitations and Future Work

  • Profiling and Migration Granularity: The current implementation uses a 4KB page granularity for profiling and migration. While this offers a good balance, the paper suggests exploring smaller granularities for more flexibility or larger ones to reduce remapping overhead. The possibility of using multiple granularities is also mentioned.
  • CXL 3.0 Features: While the paper discusses CXL 3.0's cascaded switches, the current implementation and evaluation primarily focus on a two-tier memory system. Future work could explore more complex CXL topologies.
  • Support for CXL Memory Pooling and Sharing: The paper notes that HeteroMem's transparency makes it a suitable plug-in for memory pooling and sharing scenarios, suggesting this as a future direction.
  • Hardware Overhead and Scalability: While the hardware overhead is noted as stable and minimal, further analysis on very large-scale CXL deployments could be beneficial.

Summary updated

BibTeX
@inproceedings{chen2025exploring,
  author = {Yiqi Chen and Xiping Dong and Zhe Zhou and Zhao Wang and Jie Zhang and Guangyu Sun},
  title = {{Exploring Memory Tiering Systems in the CXL Era via FPGA-based Emulation and Device-Side Management}},
  booktitle = {Pact},
  year = {2025},
  doi = {10.1109/pact65351.2025.00018},
}

Loading related content...

Figure