Abstract
NeuraLUT-Assemble is a novel framework designed to construct high-accuracy neural networks for FPGAs by assembling large "logical" neurons from hierarchical trees of smaller lookup tables (LUTs). By embedding skip-connections directly into these Boolean structures, the framework achieves up to an 8.42× reduction in area-delay product compared to state-of-the-art methods, effectively closing the accuracy gap between lookup-table-based models and traditional dense multi-layer perceptrons.
Introduction
Deploying neural networks on Field-Programmable Gate Arrays (FPGAs) often involves using Lookup Tables (LUTs) to represent neurons. While a LUT can theoretically represent any Boolean function, its resource cost grows exponentially with the number of inputs (fan-in). This "fan-in vs. resource" tension has historically forced LUT-based networks to use extremely sparse connections, which severely limits model accuracy. NeuraLUT-Assemble addresses this challenge by moving away from flat LUT structures toward a hierarchical, hardware-aware assembly approach that maintains high connectivity with ultra-low latency.
Methodology
The NeuraLUT-Assemble framework introduces several key innovations to optimize the mapping of neural networks to FPGA hardware:
- Tree-Structured Neurons: Instead of a single monolithic LUT, the framework assembles a "Logical-LUT" (L-LUT) using a tree of smaller, synthesizable units. This allows for higher connectivity without the exponential hardware penalty.
- Internal Skip-Connections: To mitigate the vanishing gradient problem during training, skip-connections are integrated within the L-LUT structures. Crucially, because these sub-networks are eventually converted into a single Boolean truth table, these skip-connections provide training benefits but carry zero hardware overhead at inference time.
- Hardware-Aware Pruning: A structured pruning strategy guides the selection of connections based on hardware constraints, ensuring the final tree structure is optimized for efficiency.
End-to-End Toolflow
The implementation process is fully automated, bridging high-level deep learning frameworks with hardware synthesis tools. The process begins in PyTorch with Quantization-Aware Training (QAT) and structured pruning. The resulting sub-networks are then converted into L-LUT truth tables, which are used to generate Verilog RTL for final implementation via Xilinx Vivado.
Experimental Evaluation
The framework was evaluated on three low-latency benchmarks: MNIST (handwritten digit classification), Jet Substructure Classification (JSC) for high-energy physics, and Network Intrusion Detection (NID).
Key Results
NeuraLUT-Assemble consistently outperformed prior state-of-the-art LUT-based models (such as LogicNets and PolyLUT) across several metrics:
- Area-Delay Product (ADP): On the MNIST dataset, the method achieved a 62× reduction in ADP compared to the original NeuraLUT while simultaneously increasing test accuracy.
- Accuracy: The models reached accuracies within 1% of full-precision, floating-point MLP models, proving that LUT-based networks can be highly expressive.
- Efficiency and Speed: For the JSC task, it demonstrated a 26× reduction in ADP. The designs achieved maximum frequencies ($F_{max}$) often exceeding 1 GHz, enabling nanosecond-level inference.
Discussion and Future Work
A primary insight of this work is the "zero-cost" skip connection; by embedding the skip-path into the Boolean function of the LUT tree, the complexity is absorbed into the truth table, making it invisible to the hardware. Furthermore, the results suggest that the topology of the tree (connectivity) is often more critical for accuracy than the raw size of the LUTs.
Future research will focus on managing pipelining complexity as tree structures deepen and exploring automated Neural Architecture Search (NAS) to optimize the massive hyperparameter space. Additionally, hybrid architectures combining different types of LUT-based neurons may offer further optimizations for the trade-off between expressiveness and hardware cost.
Summary updated