Neural architecture codesign for fast physics applications

Jason Weitz, Dmitri Demler, Luke McDermott, Nhan Tran, Javier M. Duarte

Abstract

Modern physics experiments, ranging from high-energy particle collisions to X-ray diffraction microscopy, generate massive data streams that require real-time processing at the "edge." Field-Programmable Gate Arrays (FPGAs) are the standard hardware for meeting these ultra-low latency requirements, yet designing neural networks that fit within their strict resource constraints remains a significant challenge. This paper introduces a two-stage Neural Architecture Codesign (NAC) pipeline that automates the creation of hardware-optimized models. By combining multi-objective architecture search with aggressive quantization and pruning, the framework generates models that drastically reduce computational overhead while maintaining high accuracy. We demonstrate the efficacy of this approach on two distinct physics tasks, achieving inference latencies as low as 70 nanoseconds.

Introduction

In contemporary experimental physics, sensors produce data at rates far exceeding the capacity of traditional storage and offline processing. To handle this, researchers deploy machine learning models directly onto FPGAs to perform real-time data reduction or classification. However, the design process is fraught with difficulty: a model must be small enough to fit in FPGA memory, fast enough to meet microsecond-scale triggers, and accurate enough to preserve scientific integrity. This typically requires a rare intersection of expertise in machine learning and hardware engineering.

To bridge this gap, we propose an automated framework for Neural Architecture Codesign. The goal is to allow domain scientists to deploy state-of-the-art ML on FPGAs without manual hardware optimization, moving from raw data to synthesized hardware code through an automated optimization pipeline.

Methodology

The NAC pipeline operates in two distinct stages to balance broad exploration with fine-grained optimization:

The first stage employs a multi-objective Neural Architecture Search (NAS) to explore a modular search space of layer types and sizes. Using the NSGA-II genetic algorithm, the system identifies a Pareto-optimal front of models that balance task performance against computational cost. To maintain hardware-agnosticism during this phase, we utilize Bit Operations (BOPs) as a proxy metric for hardware efficiency.

2. Local Search and Compression

Once candidate architectures are identified, they undergo a local search phase for refinement. Hyperparameters are optimized using Tree-structured Parzen Estimators (TPE). Simultaneously, the models are subjected to Quantization-Aware Training (QAT) and iterative, unstructured magnitude-based pruning. This process, facilitated by tools like Brevitas and QKeras, shrinks the model's footprint by reducing bit-precision and removing redundant weights.

3. Hardware Synthesis

The final optimized models are translated from PyTorch or TensorFlow into High-Level Synthesis (HLS) C++ code using the hls4ml library. This allows for direct deployment onto FPGAs, completing the path from high-level model design to hardware implementation.

Experimental Evaluation

We evaluated the NAC framework on two representative physics case studies:

Bragg Peak Finding (Materials Science)

This task involves predicting the $x$ and $y$ coordinates of Bragg peak centers from $11 \times 11$ pixel patches of X-ray diffraction patterns. Traditionally, this requires computationally expensive pseudo-Voigt profile fitting. Our pipeline sought to replace this with a lightweight regression model.

Jet Classification (Particle Physics)

In high-energy physics, "jets" are unordered sets of particles produced in high-energy collisions. We used a dataset where each jet contains up to 8 constituent particles characterized by transverse momentum ($p_T$), pseudorapidity ($\eta$), and azimuthal angle ($\phi$). The task is to classify the jet's origin (e.g., light quark, gluon, W/Z boson, or top quark) using a permutation-invariant "Deep Sets" architecture.

Results and Insights

Performance Benchmarks

  • Bragg Peak Task: The pipeline discovered a "Small" model that achieved a 39.2x reduction in BOPs and 2.9x fewer parameters compared to the hand-crafted BraggNN baseline, with only a marginal 3% increase in error. Synthesized for a Xilinx Virtex UltraScale+ U250 FPGA, it achieved a latency of 4.92 $\mu$s while utilizing less than 10% of the board's resources.
  • Jet Classification Task: The "Medium" Deep Sets model actually improved accuracy by 1.06% while cutting BOPs by 7.2x. For extreme latency needs, a "Tiny" model was generated that sacrificed 2.8% accuracy for a 30.25x reduction in BOPs. This model achieved a blistering inference latency of 70 nanoseconds on a Xilinx VU13P FPGA.

Key Findings

Our automated search revealed several critical insights into physics-ML models:

  1. Redundancy in Attention: In the Bragg peak task, the pipeline discovered that the non-local attention blocks used in original hand-crafted models were unnecessary; removing them maintained performance while significantly reducing complexity.
  2. Extreme Compressibility: Physics models in these domains are highly resilient to compression, supporting up to 80% sparsity and 8-bit quantization before significant performance degradation occurs.
  3. Proxy Metric Limitations: While BOPs are an effective proxy for early-stage search, they do not perfectly correlate with final FPGA resource utilization, suggesting a need for more hardware-aware metrics in future iterations.

Conclusion

The Neural Architecture Codesign pipeline successfully automates the production of high-performance, low-latency models for edge deployment in physics. By reducing the reliance on manual hardware-ML expertise, this framework enables faster iteration and deployment of advanced algorithms in experimental facilities. Future work will focus on integrating hardware-aware surrogate models to better predict inference time during the search phase and expanding the search space to include more diverse architectural components.

Summary updated

BibTeX
@article{weitz2025neural,
  author = {Jason Weitz and Dmitri Demler and Luke McDermott and Nhan Tran and Javier M. Duarte},
  title = {{Neural architecture codesign for fast physics applications}},
  journal = {Mach. Learn. Sci. Technol.},
  year = {2025},
  doi = {10.1088/2632-2153/adede1},
}

Loading related content...

Figure