TL;DR
This paper introduces SO(DA)², an end-to-end framework that automates the design and generation of specialized reconfigurable architectures (CGRAs) for data analytics. It tackles the complexity of designing custom hardware for diverse and demanding data-intensive applications by integrating software and hardware co-design.
What Problem Does This Paper Address?
Modern data analysis applications, spanning fields from scientific instruments to autonomous systems, involve complex workflows with diverse algorithms (e.g., signal processing, machine learning, graph algorithms). These applications demand extremely low latency for real-time decision-making, despite handling massive volumes of data with varying structures and velocities. While Coarse-Grained Reconfigurable Arrays (CGRAs) offer a promising balance of flexibility and efficiency compared to ASICs and FPGAs, designing and optimizing them for specific applications is a significant challenge. This requires a holistic approach that simultaneously considers software optimizations and hardware generation, which is precisely what SO(DA)² aims to provide.
Methods
SO(DA)² is a modular, open-source toolchain that automates the hardware/software co-design process. It comprises two main components:
- SODA-OPT: A high-level compiler frontend that takes applications written in high-level programming frameworks (or even C) and translates them into an intermediate representation (IR). It performs hardware/software partitioning and architecture-independent optimizations, leveraging the MLIR framework to manage various dialects representing different levels of abstraction.
- OpenCGRA: A hardware generator that takes the optimized IR and automatically explores the design space to generate specialized CGRA architectures. This includes modeling, testing, evaluation, and mapping of application kernels onto the CGRA. OpenCGRA utilizes a parameterized architectural template, allowing customization of tile count, interconnect topology, memory sizes, and functional units.
A key aspect of SO(DA)² is its integration of partial dynamic reconfiguration. This allows the CGRA to adapt its configuration at runtime, rebalancing resources to optimize performance for data streaming applications with highly variable execution times. The framework also incorporates a Design Space Exploration (DSE) engine that uses simulated annealing to find optimal trade-offs between hardware and software parameters.
Datasets
The paper evaluates SO(DA)² using two representative data-dependent applications:
- 2-layer Graph Convolutional Network (GCN): Used for protein function prediction on the ENZYMES dataset (600 graphs).
- Lower-Upper (LU) Decomposition: Applied to sparse matrices from the University of Florida sparse matrix collection (150 matrices).
These applications were chosen for their data-dependent execution times, making them suitable for demonstrating the benefits of dynamic reconfiguration.
Key Insights and Ideas
- End-to-End Automation is Crucial: The complexity of CGRA design necessitates a fully automated toolchain that bridges the gap from high-level software descriptions to specialized hardware.
- Partial Dynamic Reconfiguration is a Game-Changer for Streaming Data: For applications with highly variable data rates and kernel execution times, the ability to dynamically reconfigure parts of the CGRA at runtime significantly improves throughput and reduces latency, outperforming static partitioning.
- Hardware-Software Co-design is Not Optional: Achieving optimal performance requires simultaneous optimization of both the software mapping and the underlying hardware architecture. SO(DA)²'s DSE engine explores this joint space.
- CGRA Scalability Has Nuances: While more tiles generally offer more potential, simply increasing the number of tiles doesn't always lead to proportional speedups. Factors like loop-carried dependencies and routing complexity can limit scalability, suggesting that larger CGRAs are better suited for applications with high parallelism or for running multiple smaller applications concurrently.
- Memory Hierarchy Matters: The size of the on-chip memory (SPM) significantly impacts performance. Even a 4x reduction in SPM size only led to a ~2x speedup reduction, indicating that DSE can effectively adapt kernel tiling and unrolling decisions to smaller memory footprints.
Key Results
The paper demonstrates significant improvements using SO(DA)²:
- Speedup: The case study shows that dynamically reconfigurable CGRA designs can achieve speedups of up to 23.8x for the GCN application and 23.4x for LU decomposition compared to a baseline.
- Throughput Improvement: Dynamic rebalancing of the pipeline using partial dynamic reconfiguration led to normalized throughput improvements of up to 1.8x for GCN and 1.6x for LU, compared to a statically partitioned design.
- Reduced Latency: Dynamic reconfiguration also reduces latency, enabling faster real-time decision-making.
- Area and Power Efficiency: The generated CGRA designs are evaluated for timing, area, and power. A 5x5 CGRA design achieved an operating frequency of 800MHz @ 45nm with an average power consumption of 564.8mW. The partial dynamic reconfiguration controller occupied only 16.34% of the total area.
Limitations and Future Work
The authors acknowledge several areas for future development:
- Scalability to Larger Systems: Exploring the impact of more advanced technology nodes, larger die areas (wafer-scale), and chiplet-based approaches on design scalability.
- Integration with 3D-Stacked Memory: Investigating the applicability of CGRA designs for inclusion in 3D-stacked memory devices, potentially at conservative technology nodes.
- Enhanced DSE Heuristics: While simulated annealing is used, the authors plan to explore more effective heuristic search algorithms, including bio-inspired heuristics (evolutionary algorithms, ant colony optimization) and reinforcement learning, especially as the design space grows.
- Advanced Runtime Monitoring: Expanding runtime monitoring beyond performance to include metrics like energy consumption and real-time deadlines, enabling more sophisticated online adaptation strategies.
- Broader Application Domains: While focused on data analytics, extending the framework to other domains with similar computational demands.
Summary updated