Rhea: a Framework for Fast Design and Validation of RTL Cache-Coherent Memory Subsystems

Davide Zoni, Andrea Galimberti, Adriano Guarisco

TL;DR

This paper introduces Rhea, a unified framework that significantly accelerates the design and validation of RTL cache-coherent memory subsystems. By tightly integrating RTL generation with a hybrid system-level simulation approach, Rhea allows designers to efficiently explore architectural trade-offs and gain early insights into performance and correctness using realistic workloads.

What Problem Does This Paper Address?

Designing and validating efficient cache-coherent memory subsystems for modern multi-core System-on-Chip (SoC) architectures is a complex and time-consuming challenge. Traditional approaches often involve either high-level simulators with limited accuracy or low-level RTL simulations that are too slow for comprehensive validation with real-world applications and operating systems. This gap makes it difficult for designers to make informed early-stage design decisions, evaluate functional correctness, and analyze performance, power, and timing characteristics without significant effort and expertise. Existing solutions for FPGA-based prototyping offer higher fidelity but are often ad-hoc and require substantial hardware investment and specialized knowledge. Hybrid simulation techniques exist but are often tailored for accelerators and lack dedicated support for the intricate interactions of cache coherence subsystems.

Methods

Rhea tackles this problem with a two-pronged approach:

  1. Configurable RTL Design Flow: Rhea generates synthesizable RTL for cache-coherent memory subsystems. This RTL is highly configurable, allowing designers to specify parameters such as the number of cache levels (one or two), the number of cores supported, cache sizes, associativity, and coherence protocols (MSI is demonstrated). This modular design methodology streamlines the creation of diverse subsystem configurations.
  2. Hybrid RTL/System-Level Co-Simulation: This is the core innovation. Rhea integrates Verilator (an open-source RTL simulator that compiles RTL into C++ models) with gem5 (a popular full-system simulator). This allows the generated RTL cache-coherent memory subsystem to be simulated alongside a full system model in gem5, enabling the execution of real-world applications and operating systems. The co-simulation flow uses a scoreboard for result comparison and manages data exchange between gem5 and the Verilator-generated RTL model.

Additionally, Rhea includes a SystemVerilog coherence protocol tester, which is a port of gem5's Ruby random tester. This tool provides standalone RTL verification by stress-testing the coherence protocol with randomized memory access patterns, complementing application-driven validation.

Datasets

The paper evaluates Rhea using 22 multi-threaded applications from the PARSEC [33] and Splash-3 [34] benchmark suites. These applications are executed on simulated systems running the Ubuntu OS. The authors do not introduce new benchmarks but leverage established, representative workloads for their evaluation.

Key Insights and Ideas

  • Bridging the Fidelity-Accuracy Gap: Rhea effectively bridges the gap between the high-level abstraction of system simulators like gem5 and the low-level detail of RTL simulators. This allows for realistic workload execution with high-fidelity RTL simulation of the critical memory subsystem.
  • Early Visibility into Design: The framework provides designers with early visibility into both architectural and microarchitectural behavior, enabling faster iteration and more informed design decisions.
  • Scalability of the Hybrid Approach: The co-simulation approach demonstrates good scalability, with simulation overhead decreasing as the number of cores increases.
  • Practicality of RTL Design: By automating RTL generation and integrating it into a robust validation flow, Rhea makes complex RTL design and verification more accessible.

Key Results

  • Performance: The MSI-based RTL cache-coherent memory subsystems designed with Rhea showed intermediate performance compared to gem5's built-in MI, MESI, and MOESI models. Two-level cache hierarchies provided a performance improvement over single-level ones, especially at higher core counts.
  • Simulation Overhead: The hybrid gem5-Verilator co-simulation incurred a moderate simulation overhead, with slowdowns ranging from 1.6x to 2.7x compared to traditional gem5 MI simulations. This overhead decreases with an increasing number of cores.
  • Validation Effectiveness: The framework successfully validated functional correctness and evaluated performance under realistic workloads, including OS booting and multi-threaded applications.

Limitations and Future Work

  • MSI Protocol Focus: The current implementation and evaluation focus on the MSI cache coherence protocol. While the framework is designed to be extensible, further work would be needed to fully support other protocols like MESI or MOESI within the generated RTL.
  • Simulation Overhead: While moderate, the simulation overhead still exists. Future work could focus on further optimizing the co-simulation interface and Verilator compilation for even faster simulation times.
  • Power and Timing Analysis: While the paper mentions evaluating power, performance, and timing, the detailed results presented primarily focus on performance and simulation time. A more in-depth analysis of power and timing characteristics would be valuable.

The authors suggest that Rhea's open-source release will foster further research and reproducibility, implying that community contributions could address some of these limitations and expand the framework's capabilities.

Summary updated

Abstract

Designing and validating efficient cache-coherent memory subsystems is a critical yet complex task in the development of modern multi-core system-on-chip architectures. Rhea is a unified framework that streamlines the design and system-level validation of RTL cache-coherent memory subsystems. On the design side, Rhea generates synthesizable, highly configurable RTL supporting various architectural parameters. On the validation side, Rhea integrates Verilator's cycle-accurate RTL simulation with gem5's full-system simulation, allowing realistic workloads and operating systems to run alongside the actual RTL under test. We apply Rhea to design MSI-based RTL memory subsystems with one and two levels of private caches and scaling up to sixteen cores. Their evaluation with 22 applications from state-of-the-art benchmark suites shows intermediate performance relative to gem5 Ruby's MI and MOESI models. The hybrid gem5-Verilator co-simulation flow incurs a moderate simulation overhead, up to 2.7 times compared to gem5 MI, but achieves higher fidelity by simulating real RTL hardware. This overhead decreases with scale, down to 1.6 times in sixteen-core scenarios. These results demonstrate Rhea's effectiveness and scalability in enabling fast development of RTL cache-coherent memory subsystem designs.
BibTeX
@inproceedings{zoni2025rhea,
  author = {Davide Zoni and Andrea Galimberti and Adriano Guarisco},
  title = {{Rhea: a Framework for Fast Design and Validation of RTL Cache-Coherent Memory Subsystems}},
  booktitle = {ICCAD},
  year = {2025},
}
Figure