Formally Checking the Trigger: Proving Rare-Signal Firmware Never Drops What It Was Built to Keep

Rare-event experiments, such as searches for neutrinoless double-beta decay or heavy-flavor triggers at colliders, increasingly run their signal-versus-background classifiers as firmware on FPGAs (reconfigurable chips that run custom digital logic). A classifier running in firmware decides in microseconds which events are kept and which are thrown away for good. That makes the firmware a zero-tolerance artifact in the same sense as hand-written chip designs. A single overflowing fixed-point register or a misordered assignment can silently discard real signal, and because rejected events are never stored, nothing downstream can catch the loss. Today this firmware is produced by automated compilers and checked by simulating it on calibration data. In these experiments the true signal can never be sampled directly, so calibration data is only a stand-in for the signal, and simulation on that stand-in covers poorly the inputs that matter most.

We propose to treat trigger firmware the way chip designers treat safety-critical hardware. The plan is to state machine-checkable properties about it ("decides within the latency budget," "never stalls," "its keep/reject decision equals the reference model's bit-exactly for every input in the signal-like region") and prove them with a formal verification engine instead of sampling. A language model writes the properties. The new ingredient is how we handle the large fraction of model-written properties that turn out to be false. Each counterexample the prover returns is replayed through the executable software reference model. That replay separates real firmware bugs from wrong properties, which is the same signal-versus-background sorting problem the firmware itself solves.

Core Claim

Simulation on calibration data misses a specific and dangerous class of firmware bugs: those that change decisions only on signal-like inputs absent from the calibration set. One example is an arithmetic overflow that appears only at energies near the decay endpoint, where no labelled signal exists. Formal property checking, with properties written by a language model and false properties filtered out by reference-model replay, catches these bugs at a rate simulation cannot match at equal compute. It does so with property-authoring effort low enough to rerun every time the classifier is retrained and recompiled.

Grounding in Prior Work

The motivating system is Intelligent experiments through real-time AI. It compiles graph neural networks to an Alveo U280 FPGA with hls4ml, a tool that translates trained Python networks into hardware description code, and meets a 10 μs budget (8.82 μs edge-classification latency, 23-fold purity gain) while keeping only about 10% of data offline. It checks correctness only by simulation. Its authors name toolchain fragility as the main bottleneck, and it does no property checking at all. We reuse its compile path and add the verification layer it lacks.

The application model is the germanium classifier of Deep learning based pulse shape discrimination for germanium detectors. It compresses a 256-sample waveform into 7 numbers with an autoencoder and then applies a 141-parameter classifier, reaching 90% background rejection at 90% signal acceptance. It was trained on thorium calibration peaks precisely because true decay signal is unavailable. That training choice is exactly why calibration-only testing of its firmware is weak.

We also verify the analytic cuts of α-event characterization and rejection in point-contact HPGe detectors, which combines delayed charge recovery with amplitude-over-energy to reject surface alphas at 0.2% bulk signal loss. Analytic cuts are firmware too, and so are not exempt from bugs.

On the code-correctness side, AssertionBench both constrains and supplies our method. It checks model-written SystemVerilog Assertions (formal properties embedded in hardware code) with the JasperGold formal engine and finds an accuracy ceiling of about 44%, up to 63% of assertions refuted by counterexample, and up to 33% syntactically invalid. Are LLMs Ready for Practical Adoption for Assertion Generation? reinforces that picture. A pipeline that trusts model-written properties would therefore drown in false alarms, which is why triaging counterexamples is the center of our design rather than an afterthought. Feedback-driven generators such as VeriCoder, Automatically Improving LLM-based Verilog Generation using EDA Tool Feedback and QiMeng-CodeV-R1 supply the repair-loop technique. They repair code against tests, whereas we repair properties against proofs, with the reference model as the referee. BetterV shows that guided decoding can cut circuit size and verification runtime. It optimizes how the design is generated; we leave the design to the existing compiler and certify it.

Research Program

The system takes four inputs: the compiled register-level hardware code from hls4ml, a bit-exact fixed-point software reference of the trained classifier, a short interface specification, and a physics specification. The interface specification gives the latency budget and handshake protocol. The physics specification gives the input energy range, including the decay endpoint region, and defines the signal-like region in terms of the reference model's own inputs.

A language model drafts properties at three levels. The first covers liveness and timing: no deadlock, and a decision within N cycles. The second covers per-layer arithmetic: each neuron's register-level output equals the reference fixed-point accumulation with no unflagged overflow. The weights are hard-coded constants, so these per-neuron checks are far more tractable for a satisfiability solver than general multiplier equivalence. The third covers end-to-end decisions: for inputs within the declared range, the firmware's keep/reject bit equals the reference model's.

Every refuted property produces a concrete counterexample input trace. We replay that trace through the reference model. If the reference model also exhibits the "violating" behavior, the property was wrong: it is discarded, and the counterexample is returned to the language model to refine it. If the reference model disagrees with the firmware, the trace is a confirmed bug, together with a witness input. Syntax failures and vacuous properties are filtered with standard vacuity checks.

We run this loop for the germanium classifier, the alpha cuts, and, as a stretch target, a reduced edge classifier in the style of the sPHENIX trigger.

Baselines

The naive reference point is the current practice: simulate the compiled firmware on the calibration waveforms used for training, plus hls4ml's own co-simulation against its C model, as in Intelligent experiments through real-time AI. We give it the same wall-clock budget as our verification run. It beats us only if it detects at least as many injected bugs.

The strongest simulation competitor uses that same budget for large-scale randomized and Monte Carlo waveform testing that deliberately covers the endpoint energy region and amplitude extremes, with the reference model as the oracle. This is the alternative a physicist would actually reach for. Our method must detect strictly more of the bugs that affect only signal-like inputs to justify formal checking.

The strongest property-based competitor is a set of properties hand-written by a hardware engineer under a fixed time budget, one working day per design, checked by the same formal engine. This isolates whether the language model contributes anything beyond the verification idea itself.

The key ablation removes reference-model replay. Every counterexample is then either trusted as a bug or handed back to the model with only the prover's message. This measures how much the triage step reduces false bug reports, given AssertionBench's 63% counterexample rate.

For context on raw property quality, the best reported number in the file is AssertionBench's ceiling of roughly 44% correct assertions across models. No reported result was found on Lacuna for bug detection rates on trigger firmware, so our mutation benchmark has no prior number to beat and must be calibrated against the baselines above.

Why not just…?

Why not just keep simple analytic cuts and skip neural firmware? Analytic cuts such as amplitude-over-energy and delayed charge recovery are excellent, reaching 100% alpha rejection beyond 3 mm at 0.2% signal loss in the PPC alpha study. The neural classifier of the germanium PSD paper catches multi-site events that fool amplitude-over-energy. More importantly, analytic cuts are themselves implemented in fixed-point firmware, so our method applies to them unchanged; we include them as a verification target.

Why not just simulate much more, including synthetic signal in the endpoint region? This is the real threat. If budget-matched simulation with Monte Carlo waveforms detects the same bugs, formal checking adds little. Our mutation experiment is designed to settle exactly this, by comparing detection on bugs whose trigger region is narrow, such as overflow at a single amplitude band. Formal methods should win there because they enumerate rather than sample, but we will report the outcome either way.

Why not just use the compiler's own equivalence checking, or hand-written properties, without a language model? C-to-register equivalence certifies the compiler's translation, but not that the C model matches the trained network's fixed-point semantics across the physics range, and not latency or stall freedom. Hand-written properties may well suffice for a single 141-parameter design. The language model is justified only if property authoring must be repeated cheaply after every retrain and recompile. If the one-day engineer baseline matches the model's bug detection, we will conclude that the contribution is the property library and the replay triage, not the model.

Proposed Experiments

We will build a mutation benchmark of injected bugs, seeded into compiled firmware for the germanium classifier (trained on the HPGe pulse-shape dataset) and the alpha cuts (on the TUBE scanning alpha dataset). The bug types are bit-width truncation, accumulator overflow, blocking versus non-blocking assignment swaps (a known failure mode per VeriCoder), buffer off-by-one errors, and handshake stalls. We label each mutant by whether it changes decisions only on inputs outside the calibration set, using exhaustive reference-model comparison where tractable.

The primary metric is the escaped signal-loss rate: the fraction of mutants that alter keep/reject decisions on signal-like inputs yet pass the check. Secondary metrics are false bug reports per design, the fraction of model-written properties proven, refuted or vacuous (compared directly with AssertionBench's rates), and proof time against the latency budget. Following rare-event practice, when no escapes are observed we will report a statistical upper limit on the escape rate rather than claiming zero.

The decisive check is the comparison against budget-matched synthetic simulation on narrow-trigger mutants. Formal checking should cut escapes there substantially, while matching simulation on broad bugs that any test finds. A second decisive check is the replay ablation, which should cut false bug reports to near zero without losing confirmed bugs. A third check reruns the whole pipeline after retraining and recompiling the classifier and measures the human minutes needed. If that time is not well below the one-day engineer baseline, the case for the language model fails.