TL;DR
This paper introduces an end-to-end pipeline that compiles Symbolic Regression (SR) equations into FPGA firmware using the hls4ml framework. By replacing a 3-layer deep neural network with compact algebraic expressions, the authors achieved up to a 13x reduction in inference latency (down to 5 ns) while retaining over 90% accuracy for a critical real-time particle physics task.
What Problem Does This Paper Address?
High-energy physics (HEP) experiments, such as those at the CERN Large Hadron Collider (LHC), generate data at staggering rates. They process proton-proton collisions at 40 MHz, producing tens of terabytes of raw data per second. To handle this firehose, real-time "trigger systems" must filter data at the edge with an ultra-low maximum latency of $O(1)$ microsecond.
Field-Programmable Gate Arrays (FPGAs) are ideal for this environment. However, while deep neural networks (DNNs) offer high accuracy, deploying them on resource-constrained FPGAs is challenging due to their large memory and computational footprints. Finding the optimal trade-off between physics performance and hardware resource consumption remains a major hurdle.
Methods
Instead of traditional deep learning, the authors propose using Symbolic Regression (SR). SR searches the mathematical equation space to find simple analytic expressions that fit a dataset.
The key components of their novel end-to-end procedure include:
- PySR + hls4ml Integration: They use PySR (an open-source tool driven by an evolutionary algorithm) to discover equations, and extend the
hls4mllibrary to parse these expressions into High-Level Synthesis (HLS) code for Xilinx FPGAs. - Lookup Table (LUT) Approximations: To avoid expensive hardware implementations of complex functions (like
sin,tan,log, andexp), they implement LUT-based approximations directly into the HLS pipeline. - Latency-Aware Training (LAT): Instead of treating all mathematical operations equally during the evolutionary search, they penalize operations based on their actual clock-cycle cost on an FPGA (e.g.,
tancosts significantly more thansin).
Datasets
The method is validated on a standard high-energy physics benchmark: multiclass jet tagging using a publicly available LHC dataset from Zenodo. The task involves classifying simulated particle jets into five categories (quarks, gluons, W bosons, Z bosons, or top quarks) based on 16 physics-motivated expert features.
Key Insights and Ideas
- Direct Pareto Optimization: Unlike neural networks where architecture search is discrete and massive, SR naturally produces a Pareto front of models. This allows engineers to directly optimize and choose the best performance-to-resource trade-off.
- Hardware-Informed Evolution: By feeding FPGA clock cycle costs back into the PySR evolutionary algorithm (via LAT), the algorithm naturally discovers equations that are inherently faster to execute in hardware.
- LUTs for Speed: Replacing exact math library calls with small lookup tables drastically cuts down latency and resource usage with negligible impact on physics performance.
Key Results
- Inference Latency: The SR models reduced execution time down to just 5 ns (1 clock cycle) for exponential and logarithmic models using LUTs, compared to 65 ns (13 clock cycles) for the baseline neural network—a 13-fold speedup.
- Accuracy: The symbolic models successfully approximated the baseline 3-layer fully connected neural network, maintaining over 90% relative accuracy across various bit widths.
- Resource Reduction: The implementation drastically minimized the utilization of Digital Signal Processors (DSPs) and hardware LUTs compared to the baseline network.
Limitations and Future Work
- High-Dimensional Scaling: Evolutionary algorithms for symbolic regression scale poorly to high-dimensional datasets. To mitigate this, the authors had to pre-select 6 out of the 16 available features using a random forest regressor before running PySR.
- Future Directions: The authors suggest further optimization of the performance-resource trade-off and exploring other application domains beyond jet tagging where ultra-fast FPGA inference is required.
Summary updated