LEGO: Spatial Accelerator Generation and Optimization for Tensor Applications

Yujun Lin, Zhekai Zhang, Song Han

Abstract

As deep learning models evolve toward multi-modal foundation architectures, the demand for specialized hardware accelerators has intensified. Traditional hardware design methodologies rely heavily on rigid, manual templates that struggle to adapt to diverse tensor operations. This paper introduces LEGO, a framework that automates the synthesis of spatial accelerators from high-level algorithm descriptions. By employing a relation-centric representation and graph-based optimizations, LEGO generates synthesizable RTL that significantly outperforms existing template-based generators. Experimental results demonstrate that LEGO achieves a 3.2× speedup and 2.4× better energy efficiency compared to the Gemmini baseline across a wide range of modern neural network workloads.

Introduction

The rapid diversification of AI models—spanning Large Language Models (LLMs), generative diffusion models, and complex convolutional networks—has outpaced the flexibility of current hardware design tools. Most existing spatial accelerator generators utilize hardware templates: pre-defined skeletons where designers can only tune specific parameters like buffer sizes or the number of processing elements. While these templates simplify the design space, they are inherently rigid. When a new operation (e.g., Attention or MTTKRP) does not align with the template's fixed dataflow, performance degrades significantly.

LEGO addresses this limitation by moving toward a template-free approach. It takes high-level loop nest descriptions and automatically generates optimized hardware wiring (RTL), effectively automating the "blueprinting" process that previously required intensive manual engineering.

The LEGO Framework

LEGO decomposes the hardware generation process into two primary stages: Architecture Synthesis (Front-End) and Lower-Level Optimization (Back-End).

Architecture Synthesis

To overcome the complexity of traditional polyhedral models, which often involve difficult modulo arithmetic when mapping loops to hardware coordinates, LEGO utilizes a relation-centric representation. This approach maps time and space coordinates back to the iteration domain, simplifying the analysis of data reuse patterns.

During interconnection planning, the framework solves integer linear equations to identify optimal data reuse paths. It then applies Minimum Spanning Tree (MST) and BFS-based heuristics to determine how Functional Units (FUs) should be wired. This minimizes the need for expensive multiplexers and reduces overall data movement. Furthermore, LEGO supports Dataflow Fusion, allowing it to merge the hardware requirements of disparate algorithms (such as Conv2D and Attention) into a single, reconfigurable design.

Physical and Timing Optimization

The back-end of LEGO translates high-level designs into a Detailed Architecture Graph (DAG) composed of hardware primitives. To ensure high frequency and correct operation, LEGO uses Linear Programming (LP) solvers for Delay Matching. This process calculates the precise placement of pipeline registers so that data signals arrive at the correct functional units at the exact required cycle. Additionally, the framework performs "Pin Reusing" to share input ports across different dataflows, significantly reducing the physical area footprint.

Evaluation and Results

LEGO was evaluated against a comprehensive suite of benchmarks, including kernels like GEMM and MTTKRP, and end-to-end models such as ResNet50, GPT-2, Stable Diffusion, and LLaMA-7B. It was compared against established baselines including Gemmini (template-based), AutoSA (polyhedral-based), and SODA (HLS-based).

Performance and Efficiency

LEGO-generated accelerators achieved an average 3.2× speedup and 2.4× higher energy efficiency than Gemmini. The back-end optimizations were particularly effective, yielding a 35% reduction in area and a 28% reduction in power compared to naive RTL generation.

Architectural Insights

A key innovation in LEGO is the decoupling of control from data. By propagating control signals (such as address generation) "systolically"—passing them from one unit to the next with a specific delay—LEGO achieved a 6.5× reduction in flip-flop usage compared to designs that generate control signals locally for every unit. This approach allows the hardware to scale efficiently; LEGO can generate a design for an array of 16,384 Functional Units in approximately two minutes.

Conclusion

LEGO demonstrates that automated, template-free hardware synthesis can match or exceed the quality of expert-designed accelerators like Eyeriss or NVDLA. By treating hardware generation as a series of graph-based optimization problems, LEGO provides a scalable path for designing the next generation of AI hardware. Future work will focus on automating the internal design of Functional Units and exploring non-standard memory hierarchies to further enhance performance for non-tensor applications.

Summary updated

Abstract

Modern tensor applications, especially foundation models and generative AI applications require multiple input modalities (both vision and language), which increases the demand for flexible accelerator architecture. Existing frameworks suffer from the trade-off between design flexibility and productivity of RTL generation: either limited to very few hand-written templates or cannot automatically generate the RTL. To address this challenge, we propose the LEGO framework, which targets tensor applications and automatically generates spatial architecture design and outputs synthesizable RTL code without handwritten RTL design templates. Leveraging the affine-transformation-based architecture representation, LEGO front end finds interconnections between function units, synthesizes the memory system, and fuses different spatial dataflow designs based on data reuse analysis. LEGO back end then translates the hardware in a primitive-level graph to perform lower-level optimizations, and applies a set of linear-programming algorithms to optimally insert pipeline registers and reduce the overhead of unused logic when switching spatial dataflows. Our evaluation demonstrates that LEGO can achieve $3.2 \times$ speedup and $2.4 \times$ energy efficiency compared to previous work Gemmini, and can generate one architecture for diverse modern foundation models in generative AI applications.
BibTeX
@inproceedings{lin2025lego,
  author = {Yujun Lin and Zhekai Zhang and Song Han},
  title = {{LEGO: Spatial Accelerator Generation and Optimization for Tensor Applications}},
  booktitle = {HPCA},
  year = {2025},
  url = {https://openreview.net/forum?id=18oztAz81Y},
}
Figure