Optimizing Bit-Serial Matrix Multiplication for Reconfigurable Computing

Yaman Umuroglu, Davide Conficconi, Lahiru Rasnayake, Thomas B. Preußer, Magnus Själander

TL;DR

This paper presents BISMO, an optimized bit-serial matrix multiplication overlay for FPGAs that offers significant improvements in performance and energy efficiency. By redesigning the core dot product unit and introducing a parallel-to-serial accelerator, BISMO achieves a peak performance of 15.4 binary TOPS with 2.1 TOPS/W on an Ultra96 board, outperforming previous implementations and offering a flexible solution for applications with varying precision requirements.

What Problem Does This Paper Address?

Matrix multiplication is a fundamental operation in many scientific and engineering applications. While reduced-precision computations can significantly boost performance and energy efficiency, fixed-precision hardware is inefficient when precision requirements vary. Traditional bit-serial approaches offer flexibility but can suffer from high latency. This paper addresses the challenge of creating a high-performance, energy-efficient, and flexible matrix multiplication accelerator for FPGAs that can adapt to varying precision needs without the overhead of multiple fixed-precision instances or dynamic reconfiguration.

Methods

The authors introduce an improved version of their BISMO (Bit-Serial Matrix Multiplication Overlay) architecture. Key innovations include:

  • Optimized Dot Product Unit (DPU): A new DPU design replaces the expensive barrel shifter with a constant one-left-shift and multiplexer, and employs a more LUT-efficient compressor architecture for the AND-popcount operation. This significantly reduces resource usage and improves clock frequency.
  • Wavefront Scheduling: The DPU architecture is reorganized to process matrix multiplications in wavefronts, eliminating the need for a variable barrel shifter.
  • Parallel-to-Serial (P2S) Accelerator: A new accelerator is introduced to efficiently convert conventional bit-parallel matrices (common in CPUs) into the bit-serial format required by BISMO, addressing a critical data layout conversion bottleneck.
  • Updated Cost Model: A refined cost model is provided to estimate FPGA resource usage (LUTs and BRAMs) based on configuration parameters, enabling better design space exploration.

Datasets

Not discussed in the paper. The evaluation uses synthetic workloads with varying dimensions and bitwidths.

Key Insights and Ideas

  • Bit-serial flexibility is key: The paper reinforces that bit-serial computation is crucial for handling variable precision requirements in matrix multiplication, a common scenario in modern applications.
  • FPGA architecture matters: The optimization of the DPU for Xilinx FPGAs, specifically by leveraging their 6-input LUTs more effectively, is a significant factor in achieving improved performance and efficiency.
  • Data layout is a critical bottleneck: The introduction of the P2S accelerator highlights the importance of efficient data format conversion for bit-serial accelerators, especially when interfacing with standard CPU memory formats.
  • Tradeoffs in FPGA design: The work demonstrates clear tradeoffs between LUTs and BRAMs when scaling DPA dimensions, providing valuable insights for designers balancing resource constraints and performance goals.

Key Results

  • Peak Performance: The improved BISMO achieves a peak performance of 15.4 binary TOPS on an Ultra96 board.
  • Energy Efficiency: The system demonstrates an energy efficiency of 2.1 TOPS/W, a 1.5x improvement over the original BISMO.
  • DPU Improvement: The new DPU offers a 1.8x improvement in LUT cost per binary operation compared to the original.
  • P2S Accelerator: The P2S accelerator is 13.8x faster than a CPU implementation for data layout transformation.
  • Large Scale Performance: Synthesis results for a Xilinx Virtex UltraScale+ VU9P show the DPA can achieve 783 binary TOPS at 500 MHz, utilizing 80% of LUTs.

Limitations and Future Work

  • Limitations:
    • The paper notes that the P2S accelerator's LUT cost could be further reduced with a more optimized interconnect.
  • The cost model's prediction error is slightly underestimated for smaller designs.
  • The scalability experiments for large DPAs do not consider the matrix buffer BRAMs required to feed the array.
  • Future Work:
    • The authors suggest that splitting binary matrices into tiles could offer greater flexibility and overlap opportunities.
  • The P2S accelerator could be further optimized for interconnect design.
  • The work could be extended to explore digit-serial architectures.

Summary updated

Abstract

Matrix-matrix multiplication is a key computational kernel for numerous applications in science and engineering, with ample parallelism and data locality that lends itself well to high-performance implementations. Many matrix multiplication-dependent applications can use reduced-precision integer or fixed-point representations to increase their performance and energy efficiency while still offering adequate quality of results. However, precision requirements may vary between different application phases or depend on input data, rendering constant-precision solutions ineffective. BISMO, a vectorized bit-serial matrix multiplication overlay for reconfigurable computing, previously utilized the excellent binary-operation performance of FPGAs to offer a matrix multiplication performance that scales with required precision and parallelism. We show how BISMO can be scaled up on Xilinx FPGAs using an arithmetic architecture that better utilizes 6-LUTs. The improved BISMO achieves a peak performance of 15.4 binary TOPS on the Ultra96 board with a Xilinx UltraScale+ MPSoC.
BibTeX
@article{umuroglu2019optimizing,
  author = {Yaman Umuroglu and Davide Conficconi and Lahiru Rasnayake and Thomas B. Preußer and Magnus Själander},
  title = {{Optimizing Bit-Serial Matrix Multiplication for Reconfigurable Computing}},
  journal = {ACM Trans. Reconfigurable Technol. Syst},
  year = {2019},
  doi = {10.1145/3337929},
}
Figure