Low Bitwidth Arithmetic and Dataflow Acceleration on Reconfigurable Hardware

59 concepts56 papers1 proposal

Deep learning models are notoriously power-hungry, largely due to their reliance on high-precision floating-point math (such as FP32). When executing billions of operations on edge devices or within strict thermal budgets, standard general-purpose processors waste significant energy simply managing the overhead of floating-point representations.

To break through this efficiency barrier, a prominent research direction focuses on a hardware-algorithm co-design strategy: replacing complex floating-point arithmetic with hardware-efficient, low-bitwidth alternatives, and mapping these operations onto specialized, reconfigurable dataflow architectures like Field-Programmable Gate Arrays (FPGAs).

The Logic Gate Premium: Why Integer Arithmetic Saves Energy

To understand why changing a number format saves energy, we have to look at the physical silicon. A standard floating-point multiplier must independently handle sign bits, align exponents, multiply mantissas, and normalize the final output. This requires a massive physical footprint of logic gates.

Integer operations, by contrast, strip away this structural complexity. By shifting the mathematical heavy lifting from floating-point units to integer-based logic, hardware designers can dramatically reduce the physical area and power consumption of compute cores.

For example, Hongyin Luo et al. demonstrate in Addition is All You Need for Energy-efficient Language Models that substituting expensive floating-point multipliers with an integer-based additive approximation algorithm ($\mathcal{L}$-Mul) can cut the energy cost of tensor multiplications by up to 95% without sacrificing model performance.

Similarly, Mengzhao Chen et al. show in INT v.s. FP: A Comprehensive Study of Fine-Grained Low-bit Quantization Formats that fine-grained integer formats (such as MXINT8) reduce energy consumption by roughly 37% to 38% and silicon area by over 30% compared to equivalent floating-point units.

By eliminating high-precision multipliers entirely, frameworks like WAGE, proposed by Shuang Wu et al. in Training and Inference with Integers in Deep Neural Networks, allow both training and inference to execute entirely within low-bitwidth integer boundaries (2 to 8 bits). Even complex, non-linear operations like Softmax and LayerNorm—which traditionally require floating-point precision—can be replaced with integer approximations to save milliwatts on resource-constrained edge devices, as demonstrated by Alberto Marchisio et al. with the SwiftTron architecture in SwiftTron: An Efficient Hardware Accelerator for Quantized Transformers.

Granular Scaling: Block Floating Point and Microscaling

While pure integer quantization offers massive hardware savings, it can struggle to capture the wide dynamic range of activations and gradients in complex architectures like Large Language Models (LLMs), leading to accuracy degradation. To solve this, researchers employ block-scaled arithmetic as a middle ground.

Instead of assigning a unique exponent to every single number (as in standard floating-point formats), block floating point groups numbers into vectors or blocks that share a single, common exponent. This preserves the wide dynamic range needed to handle outliers while keeping the underlying multiplication logic as simple as integer math.

Marcelo Gennari do Nascimento et al. explore this paradigm in HyperBlock Floating Point: Generalised Quantization Scheme for Gradient and Inference Computation, proving that low-precision block floating point multipliers drastically shrink hardware area while maintaining the fidelity required for full gradient training.

To unlock the benefits of these hybrid formats, hardware architectures must support them natively. Addressing this need, Gamze İslamoğlu et al. developed MXDOTP: A RISC-V ISA Extension for Enabling Microscaling (MX) Floating-Point Dot Products. By integrating a specialized hardware unit for block-scaled dot products into a RISC-V core, the authors achieved a 25x speedup and a 12.5x improvement in energy efficiency with minimal silicon area overhead, demonstrating that block-scaled arithmetic is highly cost-effective to implement in hardware.

Bypassing the Von Neumann Bottleneck via Dataflow Architectures

Optimizing arithmetic units solves only half the problem. Standard von Neumann architectures waste significant energy moving weights and activations back and forth between centralized memory banks and execution units. Dataflow architectures eliminate this "memory tax" by structuring the physical hardware layout to mirror the natural data progression of the neural network itself.

Clément Farabet et al. pioneered this approach with NeuFlow in NeuFlow: A runtime reconfigurable dataflow processor for vision, configuring specialized hardware tiles optimized for specific operations like convolutions and non-linearities. This spatial arrangement allows data to stream directly from one operation to the next, achieving an order of magnitude higher energy efficiency than high-end GPUs.

FPGAs are uniquely suited for this dataflow paradigm because their routing fabric can be physically reconfigured to match a network's specific topology. As Qi Yu demonstrates in DLAU: A Scalable Deep Learning Accelerator Unit on FPGA, leveraging hardware tiling and specialized three-stage pipelining on FPGAs can reduce power consumption to a fraction of a watt—representing a multi-hundredfold reduction in power compared to server-grade graphics processors. This streaming dataflow approach minimizes memory access overhead, which is why Petros Toupas et al. utilize it in HARFLOW3D: A Latency-Oriented 3D-CNN Accelerator Toolflow for HAR on FPGA Devices to minimize execution latency for complex spatial-temporal models.

Co-Design and Dynamic Inference

The ultimate efficiency gains occur when the neural network architecture and the underlying hardware accelerator are optimized simultaneously. Rather than treating the hardware as a fixed target, the hardware's structural constraints are embedded directly into the model's optimization space.

Xiaofan Zhang et al. demonstrate the power of this approach in Effective Algorithm-Accelerator Co-design for AI Solutions on Edge Devices, showing that parameterizing both the neural network structure and the FPGA accelerator design into a single search space yields superior energy efficiency per inference compared to general-purpose GPUs.

This co-design philosophy naturally extends to dynamic execution. Because reconfigurable hardware can implement custom conditional logic efficiently, it can execute models that adapt their computational depth at runtime based on the difficulty of the input.

In Realization of Early-Exit Dynamic Neural Networks on Reconfigurable Hardware, Anastasios Dimitriou et al. show that mapping early-exit networks onto custom parallel or pipelined FPGA architectures allows simple inputs to bypass deeper layers, yielding substantial energy savings over static execution models.

Similarly, Alexandros Kouris et al. implement a progressive inference framework in Approximate LSTMs for Time-Constrained Inference: Enabling Fast Reaction in Self-Driving Cars, utilizing iterative refinement to provide rapid, "good enough" approximations under strict real-time constraints. By combining low-bitwidth math, dataflow routing, and dynamic execution, these systems deliver high-performance AI inference within tight edge energy budgets.

Key Foundational Work

NeuFlow: A runtime reconfigurable dataflow processor for vision (2011) — contributed 1 concept elements to this direction.
Evaluating energy efficiency of floating point matrix multiplication on FPGAs (2013) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_fpga_acceleration_390276a7,
  title = {FPGA acceleration},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/fpga-acceleration/rdr_390276a7f7174b3cb119920db62fa8e4},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_fpga_acceleration_390276a7_v1,
  title = {FPGA acceleration},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/fpga-acceleration/rdr_390276a7f7174b3cb119920db62fa8e4/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake