A Cross-Domain Matrix Engine for Signal Processing and Deep Neural Net Inference

Mehdi SafarpourarXiv 2026 · arXiv

TL;DR

This paper introduces a reconfigurable matrix engine that bridges the gap between low-latency signal processing and high-throughput AI inference. By using low-cost multiplexers to transform a standard systolic array into a tensor-style engine on the fly, the design achieves up to a 7.5× latency reduction for wireless kernels and a 1.65× speedup for neural networks with minimal hardware overhead.

What Problem Does This Paper Address?

Modern mobile devices are increasingly tasked with two very different types of heavy lifting: physical-layer (PHY) signal processing (like 5G/6G MIMO detection) and Deep Neural Network (DNN) inference. Both rely heavily on matrix multiplication, but their requirements are diametrically opposed.

  • Signal Processing typically involves small, serialized matrix operations where low latency is critical.
  • DNN Inference involves massive matrices where high throughput and energy efficiency (via data reuse) are the priorities.

Traditionally, hardware designers choose between two architectures:

  1. Systolic Arrays (SA): Excellent for large matrices due to high data reuse, but they suffer from "fill-and-drain" delays that kill performance on small matrices.
  2. Tensor Units: Fast for small operations because they perform reductions in parallel, but they require massive memory bandwidth and offer poor data reuse for large workloads.

The author argues that we shouldn't have to choose one or the other, especially in resource-constrained mobile environments.

Methods

The core contribution is a reconfigurable matrix engine that can operate as either a 2D Systolic Array or a "3D" Tensor engine.

Unlike previous "reshaping" techniques that just change the dimensions of the array, this approach reconfigures the reduction dimension itself. The author uses a set of three routing fabrics (mostly 2:1 multiplexers) to repurpose the existing Multiply-Accumulate (MAC) units and registers within a 64×64 systolic array.

  • Systolic Mode: Operates as a standard output-stationary array, propagating operands across rows and columns.
  • Tensor Mode: Reorganizes the 4,096 MACs into sixteen 16×16 planes. The existing accumulators are interconnected to form a spatial reduction tree, allowing the engine to complete small kernels in far fewer cycles.

Crucially, the design keeps the peripheral memory system and interfaces identical for both modes. This "limited bandwidth" tensor mode ensures that the hardware overhead remains low while still providing significant latency gains.

Internal routing in PE for reconfiguration

Datasets

The engine was evaluated using the SCALE-Sim simulator across two distinct workload domains:

  • Wireless PHY Kernels: Common telecom operations including Gram matrix calculation, Matched filtering, MMSE equalization, and Neumann series multiplication.
  • DNN Models: A variety of popular architectures including AlexNet, ResNet-50, GoogLeNet, MobileNet, YOLO-tiny, and DeepSpeech2.

For hardware validation, the design was verified on a Zynq 702 FPGA and synthesized using the ASAP 7nm FinFET library.

Key Insights and Ideas

  • The "Aha" Moment: The realization that the hardware resources for temporal accumulation in a systolic array can be rewired into a spatial reduction tree for a tensor engine with very little extra logic.
  • Per-Operation Reconfiguration: The mode is not fixed at boot; it can be selected by a compiler or scheduler on a per-operation basis, allowing a single chip to handle a complex pipeline of mixed matrix sizes efficiently.
  • Bandwidth Mitigation for Complex Numbers: For wireless PHY (which uses complex numbers), the author suggests generating real matrix components locally from complex inputs. This effectively halves the number of cycles needed to load new matrices in the bandwidth-limited tensor mode.
Top view of integration showing reconfiguration fabric

Key Results

  • Latency Gains: For small wireless PHY kernels, the tensor mode achieved 2.15× to 7.52× lower latency compared to a vanilla systolic array.
  • AI Speedup: On DNN workloads, the reconfigurable engine provided up to a 1.65× speedup (specifically for MobileNet), while maintaining the memory efficiency of an SA for large-batch models like ResNet-50.
  • Low Overhead: Synthesis results show that the reconfigurability adds only 13.6% to the area and 8% to the power consumption compared to a standard systolic array.
  • Energy Efficiency: The design retains the systolic array's memory access advantage for large workloads, consuming roughly 2× less energy than a pure tensor engine for ResNet-50 inference.
Wireless PHY performance comparison

Limitations and Future Work

  • Bandwidth Bottleneck: In tensor mode, the engine is still limited by the systolic array's original memory interface. While the paper shows this is often acceptable, it prevents the engine from reaching the absolute peak performance of an unrestricted tensor core.
  • Mode Selection Policy: The current selection policy is relatively simple (choosing the mode with fewer cycles). Future work could involve more sophisticated scheduling that balances performance, power, and thermal constraints.
  • Scope: The evaluation focuses primarily on matrix multiplication. While this is the dominant operation, other layers (like normalization or activation) were not the focus of this specific architectural optimization.

Summary updated

Abstract

Matrix multiplication dominates both PHY signal processing and Deep Neural Network (DNN) inference, yet these domains place opposing demands on the matrix engine. Signal processing requires low-latency execution of relatively small matrix operations, whereas DNN inference, particularly at large batch sizes, prioritizes high throughput and energy efficiency for large matrix multiplications. While offering efficient memory access profile, a Systolic Array (SA) adds fill and drain delays per matrix. Those can be amortized for large matrices but not for small matrices. Conversely, a tensor unit completes the kernel in few cycles but would require repeated reads of same operand tiles over large matrices. This makes SA too slow for signal processing and tensor too inefficient for AI. We show that this need not be a choice between systolic and tensor structuring of Multiply-Accumulate (MAC) units. We propose a single engine that operates primarily as a SA and, through low-cost reconfiguration, transforms into a tensor-style engine on a per-operation basis. MUXes create both the spatial reduction adder tree and temporal reduction accumulators from the existing MACs and registers within the systolic array. The proposed reconfigurable SA simulated using modified SCALESim simulator, verified on an Zynq 702 FPGA and synthesized using ASAP 7nm FinFET library. SCALE-Sim simulations shows up to 7.5x lower latency for small complex value PHY kernels. Even on DNN workloads, it achieves up to 1.65x speedups over vanilla SA execution with trivial increase of memory accesses, retaining the systolic array's memory access advantage. Synthesis results estimate reconfigurable fabric costs only 8% power and 13.6% area overheads.
BibTeX
@misc{safarpour2026cross,
  author = {Mehdi Safarpour},
  title = {{A Cross-Domain Matrix Engine for Signal Processing and Deep Neural Net Inference}},
  year = {2026},
  eprint = {2611.03503},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.03503},
}
Figure