SO(DA): End-to-end Generation of Specialized Reconfigurable Architectures (Invited Talk)

Antonino Tumeo, Nicolas Bohm Agostini, Serena Curzel, Ankur Limaye, Cheng Tan, Vinay Amatya, Marco Minutoli, Vito Giovanni Castellana, Ang Li, Joseph B. ManzanoPARMA-DITAM@HiPEAC 2022 · OpenReview

TL;DR

This paper introduces SO(DA)², an end-to-end framework that automates the design and generation of specialized reconfigurable architectures (CGRAs) for data analytics. It tackles the complexity of designing custom hardware for diverse and demanding data-intensive applications by integrating software and hardware co-design.

What Problem Does This Paper Address?

Modern data analysis applications, spanning fields from scientific instruments to autonomous systems, involve complex workflows with diverse algorithms (e.g., signal processing, machine learning, graph algorithms). These applications demand extremely low latency for real-time decision-making, despite handling massive volumes of data with varying structures and velocities. While Coarse-Grained Reconfigurable Arrays (CGRAs) offer a promising balance of flexibility and efficiency compared to ASICs and FPGAs, designing and optimizing them for specific applications is a significant challenge. This requires a holistic approach that simultaneously considers software optimizations and hardware generation, which is precisely what SO(DA)² aims to provide.

Methods

SO(DA)² is a modular, open-source toolchain that automates the hardware/software co-design process. It comprises two main components:

  • SODA-OPT: A high-level compiler frontend that takes applications written in high-level programming frameworks (or even C) and translates them into an intermediate representation (IR). It performs hardware/software partitioning and architecture-independent optimizations, leveraging the MLIR framework to manage various dialects representing different levels of abstraction.
  • OpenCGRA: A hardware generator that takes the optimized IR and automatically explores the design space to generate specialized CGRA architectures. This includes modeling, testing, evaluation, and mapping of application kernels onto the CGRA. OpenCGRA utilizes a parameterized architectural template, allowing customization of tile count, interconnect topology, memory sizes, and functional units.

A key aspect of SO(DA)² is its integration of partial dynamic reconfiguration. This allows the CGRA to adapt its configuration at runtime, rebalancing resources to optimize performance for data streaming applications with highly variable execution times. The framework also incorporates a Design Space Exploration (DSE) engine that uses simulated annealing to find optimal trade-offs between hardware and software parameters.

Datasets

The paper evaluates SO(DA)² using two representative data-dependent applications:

  • 2-layer Graph Convolutional Network (GCN): Used for protein function prediction on the ENZYMES dataset (600 graphs).
  • Lower-Upper (LU) Decomposition: Applied to sparse matrices from the University of Florida sparse matrix collection (150 matrices).

These applications were chosen for their data-dependent execution times, making them suitable for demonstrating the benefits of dynamic reconfiguration.

Key Insights and Ideas

  • End-to-End Automation is Crucial: The complexity of CGRA design necessitates a fully automated toolchain that bridges the gap from high-level software descriptions to specialized hardware.
  • Partial Dynamic Reconfiguration is a Game-Changer for Streaming Data: For applications with highly variable data rates and kernel execution times, the ability to dynamically reconfigure parts of the CGRA at runtime significantly improves throughput and reduces latency, outperforming static partitioning.
  • Hardware-Software Co-design is Not Optional: Achieving optimal performance requires simultaneous optimization of both the software mapping and the underlying hardware architecture. SO(DA)²'s DSE engine explores this joint space.
  • CGRA Scalability Has Nuances: While more tiles generally offer more potential, simply increasing the number of tiles doesn't always lead to proportional speedups. Factors like loop-carried dependencies and routing complexity can limit scalability, suggesting that larger CGRAs are better suited for applications with high parallelism or for running multiple smaller applications concurrently.
  • Memory Hierarchy Matters: The size of the on-chip memory (SPM) significantly impacts performance. Even a 4x reduction in SPM size only led to a ~2x speedup reduction, indicating that DSE can effectively adapt kernel tiling and unrolling decisions to smaller memory footprints.

Key Results

The paper demonstrates significant improvements using SO(DA)²:

  • Speedup: The case study shows that dynamically reconfigurable CGRA designs can achieve speedups of up to 23.8x for the GCN application and 23.4x for LU decomposition compared to a baseline.
  • Throughput Improvement: Dynamic rebalancing of the pipeline using partial dynamic reconfiguration led to normalized throughput improvements of up to 1.8x for GCN and 1.6x for LU, compared to a statically partitioned design.
  • Reduced Latency: Dynamic reconfiguration also reduces latency, enabling faster real-time decision-making.
  • Area and Power Efficiency: The generated CGRA designs are evaluated for timing, area, and power. A 5x5 CGRA design achieved an operating frequency of 800MHz @ 45nm with an average power consumption of 564.8mW. The partial dynamic reconfiguration controller occupied only 16.34% of the total area.

Limitations and Future Work

The authors acknowledge several areas for future development:

  • Scalability to Larger Systems: Exploring the impact of more advanced technology nodes, larger die areas (wafer-scale), and chiplet-based approaches on design scalability.
  • Integration with 3D-Stacked Memory: Investigating the applicability of CGRA designs for inclusion in 3D-stacked memory devices, potentially at conservative technology nodes.
  • Enhanced DSE Heuristics: While simulated annealing is used, the authors plan to explore more effective heuristic search algorithms, including bio-inspired heuristics (evolutionary algorithms, ant colony optimization) and reinforcement learning, especially as the design space grows.
  • Advanced Runtime Monitoring: Expanding runtime monitoring beyond performance to include metrics like energy consumption and real-time deadlines, enabling more sophisticated online adaptation strategies.
  • Broader Application Domains: While focused on data analytics, extending the framework to other domains with similar computational demands.

Summary updated

Abstract

Modern data analysis applications are complex workflows composed of algorithms with diverse behaviors. They may include digital signal processing, data filtering, reduction, compression, graph algorithms, and machine learning. Their performance is highly dependent on the volume, the velocity, and the structure of the data. They are used in many different domains (from small, embedded devices, to large-scale, high-performance computing systems) but in all cases they need to provide answers with very low latency to enable real-time decision making and autonomy. Coarse-grained reconfigurable arrays (CGRAs), i.e., architectures composed of functional units able to perform complex operations interconnected through a network-on-chip and configure the datapath to map complex kernels, are a promising platform to accelerate these applications thanks to their adaptability. They provide higher flexibility than application-specific integrated circuits (ASICs) while offering increased energy efficiency and faster reconfiguration speed with respect to field-programmable gate arrays (FPGAs). However, designing and specializing CGRAs requires significant efforts. The inherent flexibility of these devices makes the application mapping process equally important to the hardware design generation. To obtain efficient systems, approaches that simultaneously considers software and hardware optimizations are necessary. In this paper, we discuss the Software Defined Architectures for Data Analytics (SO(DA)²) toolchain, an end-to-end hardware/software codesign framework to generate custom reconfigurable architectures for data analytics applications. (SO(DA)²) is composed of a high-level compiler (SODA-OPT) and a hardware generator (OpenCGRA) and can automatically explore and generate optimal CGRA designs starting from high-level programming frameworks. SO(DA)² considers partial dynamic reconfiguration as key element of the system design. We discuss the various elements of the framework and demonstrate the flow on the case study of a partial dynamic reconfigurable CGRA design for data streaming applications.
BibTeX
@inproceedings{tumeo2022so,
  author = {Antonino Tumeo and Nicolas Bohm Agostini and Serena Curzel and Ankur Limaye and Cheng Tan and Vinay Amatya and Marco Minutoli and Vito Giovanni Castellana and Ang Li and Joseph B. Manzano},
  title = {{SO(DA): End-to-end Generation of Specialized Reconfigurable Architectures (Invited Talk)}},
  booktitle = {PARMA-DITAM@HiPEAC},
  year = {2022},
  url = {https://openreview.net/forum?id=dJ5BzCFkAT},
}
Figure