Table-Lookup MAC: Scalable Processing of Quantised Neural Networks in FPGA Soft Logic

Daniel Gerlinghoff, Benjamin Chen Ming Choong, Rick Siow Mong Goh, Weng-Fai Wong, Tao Luo

Abstract

Recent advancements in neural network quantisation have yielded remarkable outcomes, with three-bit networks reaching state-of-the-art full-precision accuracy in complex tasks. These achievements present valuable opportunities for accelerating neural networks by computing in reduced precision. Implementing it on FPGAs can take advantage of bit-level reconfigurability, which is not available on conventional CPUs and GPUs. Simultaneously, the high data intensity of neural network processing has inspired computing-in-memory paradigms, including on FPGA platforms. By programming the effects of trained model weights as lookup operations in soft logic, the transfer of weight data from memory units can be avoided, alleviating the memory bottleneck. However, previous methods face poor scalability - the high logic utilisation limiting them to small networks/sub-networks of binary models with low accuracy. In this paper, we introduce Table Lookup Multiply-Accumulate (TLMAC) as a framework to compile and optimise quantised neural networks for scalable lookup-based processing. TLMAC clusters and maps unique groups of weights to lookup-based processing elements, enabling highly parallel computation while taking advantage of parameter redundancy. Further place and route algorithms are proposed to reduce LUT utilisation and routing congestion. We demonstrate that TLMAC significantly improves the scalability of previous related works. Our efficient logic mapping and high degree of reuse enables entire ImageNet-scale quantised models with full-precision accuracy to be implemented using lookup-based computing on one commercially available FPGA.

Introduction: Overcoming the Memory and Scalability Walls

The "memory wall" remains a primary bottleneck in modern AI hardware. Moving weights from memory hierarchies (BRAM, DDR) to processing units like DSPs consumes significant energy and time. Computing-in-Memory (CIM) offers a solution by embedding weights directly into the hardware logic. On FPGAs, this is achieved by using Lookup Tables (LUTs)—the basic building blocks of FPGA logic—to store weights as part of the logic function itself.

However, prior attempts such as LUTNet encountered a "scalability wall." These methods were largely restricted to Binary Neural Networks (1-bit), which often lack the accuracy required for complex tasks. Scaling these approaches to higher bit-widths or larger models led to an exponential explosion in LUT requirements and insurmountable routing congestion—a "spaghetti" of interconnects that standard FPGA tools could not resolve.

Methodology: The TLMAC Framework

TLMAC (Table-Lookup Multiply-Accumulate) addresses these scalability issues through three primary innovations designed to exploit parameter redundancy and optimize resource usage.

1. Hybrid Bit-Serial Architecture

To prevent the exponential growth of logic with bit-width, TLMAC employs a hybrid architecture. It processes activations bit-serially while maintaining parallel weight processing within the LUT. This approach keeps the LUT "fan-in" (number of inputs) manageable, allowing for efficient implementation of 2-bit to 4-bit networks.

Architecture of the TLMAC Processing Element (PE) utilizing bit-serial activation processing.

2. Weight Group Clustering

Quantized networks exhibit significant redundancy; in a ResNet-18 model, unique weight patterns often constitute less than 5% of total parameters. TLMAC uses spectral clustering to identify these patterns and group them. By mapping these clusters to shared LUT resources, the framework drastically reduces the total silicon area required.

3. Routing Optimization via Simulated Annealing

To mitigate routing congestion, TLMAC utilizes a stochastic optimization technique based on simulated annealing. This algorithm intelligently places weight groups within the LUT "pool" to minimize the physical wiring needed to connect logic blocks to output accumulators, facilitating successful place-and-route on high-density designs.

Comparison of the proposed TLMAC framework against prior binary-focused LUT-based methods.

Experimental Results

The framework was evaluated using the ImageNet dataset and ResNet-18 architecture, quantized using the N2UQ (Nonuniform-to-Uniform Quantization) method.

  • Accuracy: A 3-bit ResNet-18 implemented via TLMAC achieved 71.94% accuracy on ImageNet, matching the performance of full-precision software models.
  • Resource Efficiency: TLMAC demonstrated massive savings compared to prior state-of-the-art methods like Logic Shrinkage, requiring 12x fewer LUTs for 2-bit implementations and 3x fewer for 4-bit models.
  • Scalability Milestone: This work represents the first successful implementation of an entire ImageNet-scale quantized model within the soft logic of a single, off-the-shelf FPGA (AMD Xilinx Virtex UltraScale+ XCVU13P).

Conclusion and Future Work

TLMAC shifts the Pareto frontier for FPGA-based neural network acceleration, enabling high-accuracy models to fit within standard logic footprints by treating logic as memory. While the framework successfully scales to 3-bit models, 4-bit implementations still face physical routing limits in the final layers. Future research will explore advanced floorplanning and specialized Place & Route algorithms to further push the boundaries of logic density and bit-precision in soft-logic computing.

Summary updated

Abstract

Recent advancements in neural network quantisation have yielded remarkable outcomes, with three-bit networks reaching state-of-the-art full-precision accuracy in complex tasks. These achievements present valuable opportunities for accelerating neural networks by computing in reduced precision. Implementing it on FPGAs can take advantage of bit-level reconfigurability, which is not available on conventional CPUs and GPUs. Simultaneously, the high data intensity of neural network processing has inspired computing-in-memory paradigms, including on FPGA platforms. By programming the effects of trained model weights as lookup operations in soft logic, the transfer of weight data from memory units can be avoided, alleviating the memory bottleneck. However, previous methods face poor scalability - the high logic utilisation limiting them to small networks/sub-networks of binary models with low accuracy. In this paper, we introduce Table Lookup Multiply-Accumulate (TLMAC) as a framework to compile and optimise quantised neural networks for scalable lookup-based processing. TLMAC clusters and maps unique groups of weights to lookup-based processing elements, enabling highly parallel computation while taking advantage of parameter redundancy. Further place and route algorithms are proposed to reduce LUT utilisation and routing congestion. We demonstrate that TLMAC significantly improves the scalability of previous related works. Our efficient logic mapping and high degree of reuse enables entire ImageNet-scale quantised models with full-precision accuracy to be implemented using lookup-based computing on one commercially available FPGA.
BibTeX
@inproceedings{gerlinghoff2024table,
  author = {Daniel Gerlinghoff and Benjamin Chen Ming Choong and Rick Siow Mong Goh and Weng-Fai Wong and Tao Luo},
  title = {{Table-Lookup MAC: Scalable Processing of Quantised Neural Networks in FPGA Soft Logic}},
  booktitle = {FPGA},
  year = {2024},
  doi = {10.1145/3626202.3637576},
}
Figure