Abstract
The high-energy physics community is investigating the potential of deploying machine-learning-based solutions on Field-Programmable Gate Arrays (FPGAs) to enhance physics sensitivity while still meeting data processing time constraints. In this contribution, we introduce a novel end-to-end procedure that utilizes a machine learning technique called symbolic regression (SR). It searches the equation space to discover algebraic relations approximating a dataset. We use PySR (a software to uncover these expressions based on an evolutionary algorithm) and extend the functionality of hls4ml (a package for machine learning inference in FPGAs) to support PySR-generated expressions for resource-constrained production environments. Deep learning models often optimize the top metric by pinning the network size because the vast hyperparameter space prevents an extensive search for neural architecture. Conversely, SR selects a set of models on the Pareto front, which allows for optimizing the performance-resource trade-off directly. By embedding symbolic forms, our implementation can dramatically reduce the computational resources needed to perform critical tasks. We validate our method on a physics benchmark: the multiclass classification of jets produced in simulated proton-proton collisions at the CERN Large Hadron Collider. We show that our approach can approximate a 3-layer neural network using an inference model that achieves up to a 13-fold decrease in execution time, down to 5 ns, while still preserving more than 90% approximation accuracy.
Introduction
At the CERN Large Hadron Collider (LHC), proton-proton collisions occur at a rate of 40 MHz, generating tens of terabytes of raw data per second. To manage this data volume, real-time "trigger" systems must perform machine learning inference with a strict maximum latency of approximately 1 µs. While deep learning models like neural networks (NNs) offer high accuracy, they are often resource-intensive when deployed on Field-Programmable Gate Arrays (FPGAs). Furthermore, the opaque hyperparameter space of NNs makes it difficult to optimize the trade-off between hardware resources and model performance. This work proposes Symbolic Regression (SR) as a lightweight, interpretable alternative that natively supports hardware optimization.
Methodology
We replace standard neural networks with Symbolic Regression, which uses evolutionary algorithms to discover mathematical expressions that best fit a dataset. To enable efficient FPGA deployment, we integrated several novel techniques into the hls4ml framework.
PySR and hls4ml Integration
We utilize the PySR package to generate candidate equations and extended hls4ml to parse these expressions directly into FPGA firmware. This allows for a seamless transition from mathematical discovery to hardware implementation.
LUT Approximation
Complex mathematical functions such as sin, exp, and log are computationally expensive on silicon. To address this, we implemented Lookup Table (LUT) approximations within hls4ml. These LUTs allow for the evaluation of non-linear functions in a single clock cycle, significantly reducing latency.
Latency-Aware Training (LAT)
Standard SR treats all mathematical operators as having equal complexity. We modified the PySR penalty function to account for actual FPGA clock cycle latency (e.g., a tan() operation is penalized more heavily than an addition). This Latency-Aware Training (LAT) ensures the evolutionary algorithm favors hardware-friendly equations.
Datasets and Validation
The method was validated using the LHC Jet Identification dataset. This multiclass classification task involves tagging particle "jets" as originating from quarks, gluons, W/Z bosons, or top quarks. Due to the combinatorial complexity of SR, we utilized a random forest regressor to pre-select the 6 most significant features from the original 16 physics-motivated inputs.
Results and Discussion
Our SR-based approach demonstrated significant improvements in both speed and resource efficiency while maintaining high predictive performance.
Inference Performance
The best-performing SR models achieved an inference latency of just 5 ns (1 clock cycle). This represents a 13-fold speedup compared to a baseline 3-layer Quantization-Aware Training (QAT) neural network, which required 65 ns (13 clock cycles). This extreme low latency is critical for the 40 MHz environment of the LHC.
Accuracy and Robustness
Despite the reduction in complexity, the SR models preserved over 90% relative accuracy compared to the baseline NN. As shown in the figure below, the performance remains stable across various bit widths, indicating that the symbolic expressions are robust to quantization.
Hardware Efficiency
By using SR and LUT approximations, FPGA resource usage (DSPs and LUTs) was reduced by orders of magnitude. This efficiency allows for the deployment of more complex logic or multiple algorithms on a single chip.
Conclusion
This work demonstrates that Symbolic Regression, combined with hardware-aware optimization and LUT approximations, provides a powerful alternative to traditional deep learning for ultra-low latency FPGA inference. While current evolutionary algorithms face scaling challenges with high-dimensional data, the ability to directly optimize the Pareto front of accuracy versus hardware cost is a significant advantage for edge computing in high-energy physics and beyond.
Summary updated