A Prototype-Based Framework to Design Scalable Heterogeneous SoCs with Fine-Grained DFS

Gabriele Montanaro, Andrea Galimberti, Davide Zoni

TL;DR

This paper introduces Vespa, an open-source framework that extends the ESP toolchain to enable faster design space exploration and runtime optimization of complex, heterogeneous System-on-Chips (SoCs). Vespa allows designers to easily scale accelerator performance, dynamically adjust clock frequencies across different parts of the chip, and monitor system behavior in real-time, significantly streamlining the development of advanced embedded systems.

What Problem Does This Paper Address?

Modern embedded systems, or Systems-on-Chips (SoCs), are becoming increasingly complex, often featuring a mix of general-purpose processors and specialized hardware accelerators. This complexity, coupled with the slowdown in traditional CPU performance scaling, necessitates efficient design methodologies. Existing frameworks, while helpful, often lack the flexibility needed for thorough design space exploration (DSE) and effective runtime optimization. Specifically, they struggle with:

  • Scalability: Easily increasing the performance of accelerators without redesigning them.
  • Dynamic Power Management: Efficiently managing power consumption by adjusting clock frequencies of different chip components independently.
  • Runtime Visibility: Gaining insights into the system's behavior during operation to guide optimization.

This paper addresses these challenges by proposing Vespa, a framework designed to tackle the intricate design and optimization of large, heterogeneous SoCs, particularly those implemented on FPGAs.

Methods

Vespa builds upon the existing ESP framework and introduces three core innovations:

  1. Multi-Replica Accelerator Tiles: This allows designers to instantiate multiple copies (replicas) of a hardware accelerator within a single computing tile. This approach scales the accelerator's throughput by a factor of K (the replication factor) without modifying the accelerator's design or the network-on-chip (NoC) interconnect. This is achieved by using an AXI bridge to multiplex the interfaces of the accelerator replicas.
  2. Configurable-DFS Frequency Islands: The SoC can be partitioned into "frequency islands" at design time. Each island, comprising one or more tiles and NoC routers, can operate at an independent clock frequency. These frequencies can be fixed or dynamically adjusted by dedicated Dynamic Frequency Scaling (DFS) actuators. The DFS mechanism uses two clock managers to ensure a smooth transition during reconfiguration, avoiding clock-gating effects. Resynchronizers are used at island boundaries to handle clock domain crossings.
  3. Run-time Monitoring Infrastructure: A dedicated infrastructure is implemented to collect various execution statistics from accelerators. This includes execution time, packet counts, and round-trip time (for memory accesses). These statistics are exposed through memory-mapped hardware counters, accessible via the SoC's CPU or a host interface, providing crucial data for DSE and runtime optimization.

Datasets

The paper does not explicitly mention training datasets. However, it evaluates its framework using the CHStone benchmark suite for HLS-generated accelerators. The experimental setup utilizes a Siemens proFPGA quad motherboard with an AMD Virtex-7 2000 FPGA.

Key Insights and Ideas

  • Performance Scaling without Redesign: The multi-replica accelerator tile approach is a surprisingly simple yet effective way to boost throughput. The overhead in terms of LUTs, FFs, and BRAMs is significantly less than the performance gain, making it a very efficient trade-off.
  • Granular Power Control: The concept of frequency islands allows for fine-grained control over power consumption. Different parts of the SoC can operate at speeds optimized for their current workload, rather than a uniform, potentially inefficient, clock speed for the entire chip.
  • Runtime Data is Crucial: The integrated monitoring infrastructure is not just for debugging; it's presented as a fundamental tool for design space exploration and runtime optimization. This highlights a shift towards more adaptive and data-driven SoC design.
  • FPGA Prototyping as a Standard: The paper reinforces the trend of using FPGA-based prototyping as a standard paradigm for rapid design and validation of complex heterogeneous platforms, complementing traditional simulators.

Key Results

  • Throughput Improvement: The multi-replica accelerator tiles demonstrated significant throughput increases. For 2x replication, the average throughput increase was 1.92x, and for 4x replication, it was 3.58x. This improvement was achieved with a much smaller increase in resource utilization (LUTs, FFs, BRAMs), especially compared to the performance gain. DSP resources scaled more directly with replication.
  • Resource Utilization: For baseline accelerators, resource utilization on the target FPGA was relatively low (e.g., up to 1.4% LUT, 0.6% FF). Multi-replica instances showed increased utilization, but the performance-per-resource ratio was favorable.
  • Memory-Bound vs. Compute-Bound Analysis: Experiments showed that compute-bound accelerators (like adpcm) maintained performance even under moderate NoC traffic, while memory-bound accelerators (like dfmul) experienced significant throughput degradation as NoC traffic increased, validating the framework's ability to analyze these trade-offs.
  • DFS Impact: The paper demonstrated how varying clock frequencies of different islands (e.g., Al/A2 tiles, NoC/MEM, TG cores) impacts memory traffic, showing the practical utility of DFS for optimizing system behavior.

Limitations and Future Work

  • Limitations: The paper acknowledges that the DFS actuator implementation relies on specific FPGA clock management resources (MMCMs). The performance scaling of multi-replica tiles is shown to be more linear for compute-bound accelerators than for memory-bound ones, where NoC contention can become a bottleneck. The presented results are based on a specific FPGA platform.
  • Future Work: The authors suggest that the Vespa framework is publicly available and open-source, aiming to foster further research. Potential future work could involve exploring more advanced DFS strategies, integrating more sophisticated runtime monitoring capabilities, and applying the framework to even larger and more complex SoC designs. The framework's ability to analyze the impact of DFS on local and global traffic suggests further exploration in power-aware routing and scheduling.

Summary updated

BibTeX
@inproceedings{montanaro2024prototype,
  author = {Gabriele Montanaro and Andrea Galimberti and Davide Zoni},
  title = {{A Prototype-Based Framework to Design Scalable Heterogeneous SoCs with Fine-Grained DFS}},
  booktitle = {Iccd},
  year = {2024},
  doi = {10.1109/iccd63220.2024.00108},
}

Loading related content...

Figure