SPMW: A Grid-Based Programming Model for Scalable Accelerator Design

Hongzheng Chen, Shihan Fang, Niansong Zhang, Sunwoo Kim, Weiran Liu, Zhiru ZhangarXiv 2026 · arXiv

TL;DR

SPMW (Single Program, Multiple Connected Work Units) is a new programming model for hardware accelerators that treats structural regularity—like the grids in a TPU—as a first-class citizen. By decoupling processing element (PE) logic from how those elements are connected and fed data, SPMW achieves up to 619x faster compilation and significant latency reductions compared to state-of-the-art design tools.

What Problem Does This Paper Address?

Modern AI accelerators, such as Google’s TPU or Cerebras’s Wafer-Scale Engine, achieve high performance by replicating simple processing elements (PEs) across massive, regular grids. However, existing hardware design tools struggle to handle this regularity efficiently.

High-Level Synthesis (HLS) tools often treat these grids as sequential loops, while Register-Transfer Level (RTL) languages require designers to manually wire thousands of connections. In both cases, the underlying "regularity" of the design is often lost or "flattened" during compilation. This leads to redundant work—where a compiler optimizes the same PE logic thousands of times—and makes it incredibly difficult for engineers to explore different architectures (e.g., changing a mesh to a butterfly network) without rewriting massive chunks of code.

Methods

The authors introduce SPMW, a grid-based programming model implemented with a Python frontend and an MLIR-based compiler. The core innovation is the explicit decoupling of three design concerns:

  1. Units: The "Single Program" that defines the behavior of an individual PE using named, typed ports.
  2. Topology: A separate rule that defines the logical grid and how ports are connected (e.g., "connect my east port to my neighbor's west port").
  3. Fabric: The surrounding infrastructure that handles memory banking, data loaders, and result collection.
Typed composition of units and fabrics

The SPMW compiler exploits this structure by synthesizing each unique PE configuration only once, then reusing that implementation across the entire grid. It also features a parallel functional simulator that uses OpenMP to provide rapid feedback on design correctness without waiting for slow, cycle-accurate RTL simulations.

Datasets and Benchmarks

The authors evaluated SPMW across several complex hardware designs:

  • Systolic GEMM: Comparing output-stationary and weight-stationary matrix multiplication arrays against AutoSA and Allo.
  • FFT (Fast Fourier Transform): Testing complex butterfly connection patterns against HP-FFT and Spiral.
  • Mini-TPU: A custom instruction-programmable accelerator running real-world workloads from LLaMA 3.2 1B and DeepSeek-V4-Pro.
  • FEATHER: A research accelerator with a programmable reduction network.
  • Agentic Design Study: A novel benchmark evaluating how five different LLMs (including Claude and GPT-4 variants) perform when writing hardware in SPMW vs. SystemVerilog or C++ HLS.

Key Insights and Ideas

  • Regularity as a First-Class Citizen: By retaining the knowledge that a design is a grid throughout the entire compilation flow, the tools can avoid redundant optimization and enable parallel processing of the design itself.
  • Structural Decoupling: You can change the entire dataflow of an accelerator (e.g., moving from a mesh to a grouped attention array) by editing just a few lines of topology code, without touching the core arithmetic logic of the PEs.
  • AI-Friendly Hardware Design: The study found that SPMW is significantly easier for AI "coding agents" to write. Because the language is more concise and modular, agents produced working designs with 7.51x fewer tokens than they needed for SystemVerilog, reducing errors and cost.

Key Results

  • Massive Speedups: For a 32×32 systolic array, SPMW reduced GEMM latency by 4.36x and accelerated parallel kernel-to-RTL compilation by 619x compared to AutoSA.
  • Improved Performance: SPMW-generated FFTs achieved up to 2.65x lower latency than the state-of-the-art HP-FFT baseline.
  • Faster Iteration: The parallel functional simulator provided up to an 18.4x speedup over standard Vitis HLS C-simulation for mini-TPU workloads.
  • Productivity: SPMW required up to 3.92x fewer lines of code than HLS and 18.58x fewer than RTL for complex designs like the FEATHER accelerator.

Limitations and Future Work

While SPMW excels at functional simulation and structural organization, the authors note that the current simulator is not cycle-accurate, meaning it can't perfectly predict hardware performance before synthesis. Furthermore, while the frontend is fast, the final "backend" stages of physical placement and routing (P&R) still operate on the full, flattened design and remain a time bottleneck.

Future work aims to extend SPMW to target other specialized hardware like AMD AI Engines and Cerebras chips, and to integrate formal verification tools to guarantee that these complex grids are free from deadlocks.

Summary updated

Abstract

Modern accelerators specialize recurring computation and replicate it across regular hardware structures to achieve high performance. Sustaining this parallelism requires equally deliberate data delivery. Yet design tools often express these structures as sequential programs or flatten them into separate hardware modules and connections. This obscures shared patterns and leads to repeated compiler work. We introduce Single Program, Multiple Connected Work Units (SPMW), a grid-based programming model for scalable accelerator design that makes grid regularity a first-class abstraction. SPMW decouples local work-unit behavior from grid connectivity and data delivery. To our knowledge, SPMW is the first programming model to retain regularity throughout the compilation process, combining behavioral HLS with structural RTL generation for both fixed-function and domain-specific instruction-programmable accelerators. We implement SPMW with a Python frontend and an MLIR-based compiler that synthesizes each distinct kernel configuration once for reuse across independent hardware instances. Preserving regularity also enables parallel compilation and functional simulation to accelerate iterative design feedback. We use SPMW to generate a 32x32 systolic array that reduces GEMM latency and parallel kernel-to-RTL compilation time by 4.36x and 619x relative to AutoSA. Our FFT achieves up to 2.65x lower latency than HP-FFT. Our mini-TPU and our SPMW implementation of the FEATHER accelerator demonstrate complex composition and outperform their RTL baselines. Parallel functional simulation provides up to 18.4x speedup over Vitis HLS C simulation on the mini-TPU workloads. SPMW is also agent-friendly: given the same design specification, all five coding agents produce SPMW designs that satisfy the correctness and latency requirements, with 7.51x lower token usage than for SystemVerilog.
BibTeX
@misc{chen2026spmw,
  author = {Hongzheng Chen and Shihan Fang and Niansong Zhang and Sunwoo Kim and Weiran Liu and Zhiru Zhang},
  title = {{SPMW: A Grid-Based Programming Model for Scalable Accelerator Design}},
  year = {2026},
  eprint = {2611.12944},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.12944},
}
Figure