NeuraLUT: Hiding Neural Network Density in Boolean Synthesizable Functions

Marta Andronic, George A. Constantinides

TL;DR

NeuraLUT is a novel framework that packs dense, high-precision neural sub-networks into FPGA lookup tables (LUTs), effectively "hiding" computational complexity inside synthesizable Boolean functions. This approach allows researchers to train deep, expressive models that collapse into ultra-low-latency, shallow hardware circuits, achieving up to 4.3× lower latency than previous state-of-the-art methods.

What Problem Does This Paper Address?

Deep Neural Networks (DNNs) are notoriously difficult to deploy on "the edge" (like sensors or embedded devices) because they require massive amounts of memory and computation. Field-Programmable Gate Arrays (FPGAs) are a popular solution, but traditional implementations struggle with the trade-off between accuracy and latency.

Previous works like LogicNets and PolyLUT tried to solve this by mapping individual neurons directly to FPGA Lookup Tables (LUTs). However, these methods are limited: if the network is too shallow, it isn't accurate enough; if it’s deep enough to be accurate, the hardware latency (the time it takes for data to travel through many layers of logic) becomes too high for real-time applications like particle physics or high-speed networking.

Methods

The core innovation of NeuraLUT is Sub-network Encapsulation. Instead of mapping one neuron to one LUT, the authors map an entire multi-layer sub-network into a single "Logical LUT" (L-LUT).

  1. Dense Training: During the training phase, each L-LUT is treated as a small, fully connected Multi-Layer Perceptron (MLP) using full floating-point precision.
  2. Skip Connections: To train deeper sub-networks without running into "vanishing gradient" problems, the authors include residual (skip) connections inside these partitions.
  3. The "Baking" Process: Once trained, the entire sub-network—regardless of its internal depth or complexity—is converted into a truth table. Because the inputs and outputs between partitions are quantized (low bit-width), the complex math "collapses" into a simple Boolean function that the FPGA can execute in a single step.
  4. Sparsity: To keep the LUT sizes manageable, they use a priori random sparsity, restricting each L-LUT to a small number of inputs (fan-in).

Datasets

The authors evaluated NeuraLUT on two distinct benchmarks:

  • MNIST: The classic digit classification dataset (784-dimensional inputs, 10 classes).
  • Jet Substructure Tagging: A high-energy physics dataset from CERN used to classify types of particles in the Large Hadron Collider. This is a quintessential "ultra-low latency" task where decisions must be made in nanoseconds.

Key Insights and Ideas

  • Complexity is "Free" (Inside the LUT): On an FPGA, a LUT with $K$ inputs takes the same amount of time to compute a simple "AND" gate as it does to compute a highly complex, non-linear function. NeuraLUT exploits this by cramming as much "intelligence" as possible into that Boolean function.
  • Hiding the Datapath: By keeping high-precision math and skip connections inside the sub-network, these elements never appear in the final hardware circuit. The "exposed" datapath only sees low-precision connections, which drastically reduces the routing congestion and area on the chip.
  • Linear vs. Exponential Scaling: Unlike previous polynomial-based methods (PolyLUT), NeuraLUT’s parameter count scales linearly with the number of inputs for a fixed sub-network size, making it easier to optimize.

Key Results

NeuraLUT consistently outperformed existing LUT-based and DSP-based (Digital Signal Processing) architectures:

  • Latency: On the jet tagging task, NeuraLUT achieved a latency of just 14ns, which is 1.6× faster than PolyLUT and 4.3× faster than LogicNets for similar accuracy.
  • Area Efficiency: On MNIST, NeuraLUT reached 96% accuracy with a 26× reduction in the area-delay product compared to the FINN framework.
  • Accuracy Stability: The inclusion of skip connections allowed the models to maintain high accuracy even as the sub-networks grew deeper, a feat that previous LUT-based methods struggled to achieve.

Limitations and Future Work

The primary limitation is the exponential growth of LUT size relative to input bit-width and fan-in. If a sub-network requires too many inputs or high-resolution input data, the resulting truth table becomes too large to fit on the FPGA.

The authors suggest that future work should involve Neural Architecture Search (NAS) to automatically determine the optimal "shape" of these sub-networks and the best way to connect them, further pushing the boundaries of what can be "hidden" inside a Boolean function.

Summary updated

Abstract

Field-Programmable Gate Array (FPGA) accelerators have proven successful in handling latency- and resource-critical deep neural network (DNN) inference tasks. Among the most computationally intensive operations in a neural network (NN) is the dot product between the feature and weight vectors. Thus, some previous FPGA acceleration works have proposed mapping neurons with quantized inputs and outputs directly to lookup tables (LUTs) for hardware implementation. In these works, the boundaries of the neurons coincide with the boundaries of the LUTs. We propose relaxing these boundaries and mapping entire sub-networks to a single LUT. As the sub-networks are absorbed within the LUT, the NN topology and precision within a partition do not affect the size of the lookup tables generated. Therefore, we utilize fully connected layers with floating-point precision inside each partition, which benefit from being universal function approximators, but with rigid sparsity and quantization enforced between partitions, where the NN topology becomes exposed to the circuit topology. Although cheap to implement, this approach can lead to very deep NNs, and so to tackle challenges like vanishing gradients, we also introduce skip connections inside the partitions. The resulting methodology can be seen as training DNNs with a specific FPGA hardware-inspired sparsity pattern that allows them to be mapped to much shallower circuit-level networks, thereby significantly improving latency. We validate our proposed method on a known latency-critical task, jet substructure tagging, and on the classical computer vision task, digit classification using MNIST. Our approach allows for greater function expressivity within the LUTs compared to existing work, leading to up to $4.3 \times$ lower latency NNs for the same accuracy.
BibTeX
@inproceedings{andronic2024neuralut,
  author = {Marta Andronic and George A. Constantinides},
  title = {{NeuraLUT: Hiding Neural Network Density in Boolean Synthesizable Functions}},
  booktitle = {FPL},
  year = {2024},
  url = {https://openreview.net/forum?id=NNUcl3N4MV},
}

Loading related content...

Figure