Ground-based gamma-ray telescopes photograph the faint blue flash that a particle shower makes as it crosses the atmosphere. They must decide within milliseconds whether each flash came from a gamma ray or from one of the thousands of cosmic-ray protons that mimic it. Real-time pipelines meet this deadline by using cheap shape summaries and a decision-tree classifier, and they pay heavily for it. On the Large-Sized Telescope prototype (LST-1), the real-time chain detects the blazar Markarian 421 at 15.3σ, while the slower offline chain reaches 31.2σ on the same 4.2 hours of data. Deep convolutional networks that read the raw camera pixels do better still, but they are too expensive to run on every event at trigger rates of up to 14 kHz. The obvious fix is to run the expensive network only on the events where it would change the answer. This proposal argues that doing this well is an online, time-varying optimization problem with a specific trap. The compute budget, the trigger rate, and the sky conditions all drift during a night. Worse, a router that learns only from the events it chose to escalate can lock itself into never escalating whole classes of events.
We propose a router with three parts. It predicts, from cheap features, how much the deep network would change each event's outcome. It escalates events whose predicted change exceeds a price that tracks the current compute budget. It keeps a small, randomized audit stream so that its own predictions cannot freeze into a self-confirming blind spot. The deliverable is a measured curve of sensitivity against compute for real-time Cherenkov analysis under drifting conditions. The curve is designed to show whether online adaptation and audit exploration beat a simple static cascade, or whether the static cascade is enough.
Core Claim
The claim is that a static confidence cascade cannot stay efficient through a night. By a static cascade we mean a fixed threshold, calibrated offline on simulation, on the cheap classifier's uncertainty. It loses efficiency for two reasons. First, the budget is not fixed. Trigger-rate bursts and shared compute change how many deep-network calls per second are affordable. Second, the relationship between cheap features and "the deep network would disagree here" shifts with night-sky background light, atmospheric transmission, and zenith angle. The file documents both effects. Simulation-trained networks degrade on real telescopes because of unmodeled sky background, mirror degradation and aerosol layers. The real-time pipeline's deficit is itself partly caused by simplified calibration.
An online router must learn this shifting relationship without ground truth. It can only observe disagreement between the fast path and the deep path on events it escalates. That selective feedback is where premature convergence enters. It is the same "local sensing blindness" that limits event-triggered Bayesian optimization, where drift is detected only at points the algorithm chooses to sample.
The new element is the combination, and its evaluation under realistic drift. The combination has three parts: a budget price that tracks a time-varying constraint, an escalation-gain model updated on label-free disagreement, and an explicit audit stream that keeps the gain estimate honest in regions the router has stopped visiting.
Research Program
The router sees, for each event, the cheap features already computed at the first data-reduction step: Hillas moments, integrated charge, timing slope, and the decision-tree gammaness. It also sees slow context variables: the current sky-background rate, the trigger rate, and the most recent atmospheric transmission estimate from the Raman LIDAR described in A 1.8 m Class Pathfinder Raman LIDAR for the Northern Site of the Cherenkov Telescope Array Observatory—Technical Design.
The gain target $g$ for an event is label-free. We define it as the change in science-relevant output when the deep network replaces the fast path. That means whether the gamma/hadron decision flips across the analysis cut, plus the shift in reconstructed direction measured in units of the point-spread function. A small regression model $\hat g_t(z)$ predicts this from features $z$. The escalation rule is a threshold on price-weighted gain:
$$\text{escalate if } \hat g_t(z) > \lambda_t c, \qquad \lambda_{t+1} = \big[\lambda_t + \eta\,(C_t - B_t)\big]_+ ,$$
Here $c$ is the per-event cost of the deep network, and $C_t$ and $B_t$ are the compute spent and the compute allowed in window $t$. The step size $\eta$ controls how fast the price reacts, and $[\cdot]_+$ clips the price at zero.
This is the online primal-dual pattern for time-varying constraints, taken from Simultaneously Achieving Sublinear Regret and Constraint Violations for Online Convex Optimization with Time-varying Constraints and Online Convex Optimization with Perturbed Constraints: Optimal Rates against Stronger Benchmarks. Those papers supply the method and its guarantees for convex losses. We apply it to a budget constraint that is set by the detector's own trigger stream.
The threshold structure is motivated by remote-estimation theory. There, "stay silent unless the innovation exceeds a threshold" is provably optimal over costly channels (Optimal Sensor Scheduling and Remote Estimation over an Additive Noise Channel), and the optimal threshold moves with channel state (Remote estimation over a packet-drop channel with Markovian state). Escalation plays the role of transmission, and the fast path plays the role of silence. Those results also warn us that they assume stationary dynamics, which is exactly what fails here. That is why $\hat g_t$ must adapt.
Premature convergence is the central risk. $\hat g_t$ is updated only on escalated events. So if drift makes some region valuable, for example low-charge events after the sky background rises, the router will never find out, because it never escalates events there. We add a budgeted audit stream. With small probability $\epsilon$, an event is escalated regardless of $\hat g_t$. Its gain is used with importance weight $1/\epsilon$, so the estimate stays unbiased everywhere. A global drift trigger in the spirit of Event-Triggered Time-Varying Bayesian Optimization compares audited gains against predictions. When the gap exceeds the model's error envelope, it temporarily raises $\epsilon$ and shortens the gain model's memory. Because audits are drawn uniformly, this trigger also covers regions the router has abandoned, which the pointwise trigger in that paper cannot do.
We deliberately avoid momentum in both the price and gain-model updates. On the Provable Suboptimality of Momentum SGD in Nonstationary Stochastic Optimization shows that momentum creates an inertia window of order $\kappa/(1-\beta)$ under drift. In that paper's drifting-quadratic experiment, heavy-ball error rose from 0.34 to 401.37 as β went from 0.5 to 0.99, while plain SGD stayed near 1.04. We treat this as a design constraint and test it with a momentum ablation.
The fast path is the existing real-time chain from The Real Time Analysis framework of the Cherenkov Telescope Array's Large-Sized Telescope. The deep path is one of the networks from Deep-learning-driven event reconstruction applied to simulated data from a single Large-Sized Telescope of CTA, either γ-PhysNet or the lighter CTLearn-TRN. The router itself costs microseconds, which is negligible next to the 0.5 ms/event the first reduction step already spends.
Baselines
The naive reference is the fast path alone: the current real-time chain, with decision-tree gammaness on Hillas features. Its reported result on Markarian 421 is 15.3σ at a gammaness cut of 0.75, against 31.2σ offline (The Real Time Analysis framework of the Cherenkov Telescope Array's Large-Sized Telescope). Its sensitivity is about twice as poor above 0.4 TeV. The upper reference is the deep network on every event with no compute limit. Its reported simulation result is angular resolution up to 40% better than random forests, 0.3°–0.5° at low energy and under 0.1° above 2 TeV (Deep-learning-driven event reconstruction applied to simulated data from a single Large-Sized Telescope of CTA). For its per-event latency or energy on real-time hardware, no reported result found on Lacuna, so we will measure it ourselves.
The strongest competitor is a static cascade. It uses a gain model trained offline on simulation and a fixed threshold chosen so that the average escalation fraction equals the budget. This is what a practitioner would build first. It is a real threat whenever the gain map and the budget are roughly stationary. A second strong competitor is the static gain model plus our adaptive price. This isolates whether adapting the gain model matters beyond simply tracking the budget.
Uniform random escalation at the same budget tests whether routing helps at all. The ablations are the full method without audits and without the drift trigger, which isolates the premature-convergence ingredient, and the full method with momentum in its updates.
All methods receive the same fast-path features, the same deep network weights, and the same deep-network call budget. Compute is counted both as calls per second and as measured GPU-seconds. Hyperparameters are tuned on held-out simulation nights with an equal number of configurations per method. Two results count as beating the static cascade. The first is a significantly higher fraction of the all-CNN improvement recovered, measured in angular resolution, energy resolution, and detection significance, at matched budget on drifting test nights. The second is significantly fewer budget violations during trigger bursts. Both use bootstrap intervals over nights.
Why not just…?
Why not just use a static cascade with an offline-calibrated threshold? This is the most serious alternative. If simulation-trained gain estimates transfer to real conditions and the budget is steady, it will match our method at lower complexity. The file gives reasons to doubt this but no direct evidence. Simulation-to-real shift is documented for deep reconstruction, and remote-estimation thresholds are known to shift with channel state. Our stationary-condition experiment is where the static cascade should tie us. If it also ties us under injected drift, we will report that and recommend the static cascade.
Why not just make the deep network cheap enough, through quantization, a smaller model, or more GPUs, and run it on everything? At a 14 kHz peak, a uniform deep pass multiplies compute by the entire event rate. Shrinking the model also erodes the accuracy that motivated using it. Routing and compression are complementary, so we will run the router with CTLearn-TRN as well as γ-PhysNet. If a compressed network on all events beats routed γ-PhysNet at equal GPU-seconds, that outcome decides the question in favor of compression.
Why not just escalate a uniform random subset? Random escalation never gets trapped. But it spends budget on bright, unambiguous showers where the fast path already agrees with the deep network. Santi et al. report that state-dependent transmission roughly halves channel use at 90% accuracy relative to randomized policies (Remote Estimation of Markov Processes over Costly Channels: On Implicit Information Benefits). That is a Markov-chain result, not a Cherenkov one, and the uniform baseline measures the gap directly here.
Why not just reset the gain model whenever its errors spike, as in event-triggered Bayesian optimization? A reset driven by errors on escalated events inherits the pointwise blindness of Event-Triggered Time-Varying Bayesian Optimization. It cannot detect that an abandoned region has become valuable. The no-audit ablation, combined with a reset-trigger variant, tests whether uniform audits catch shifts that resets miss. The diversity literature warns of a cost: Divergence-Guided Particle Swarm Optimization loses precision on unimodal problems because of its exploration term. Audits likewise waste budget when nothing drifts, so we will report that tax at $\epsilon$ values from 0.5% to 5%.
Proposed Experiments
The primary testbed is the LST-1 Monte Carlo benchmark from Deep-learning-driven event reconstruction applied to simulated data from a single Large-Sized Telescope of CTA. It has about 900k diffuse gamma and 900k proton events for training and 1M point-like gamma events for testing, generated with CORSIKA and sim_telarray. We will build synthetic "nights" by concatenating test events into time streams. Each stream has a scheduled drift: sky-background levels stepped and ramped within sim_telarray, atmospheric transmission profiles varied to mimic aerosol layers of the kind the Raman LIDAR measures, and trigger-rate bursts up to 14 kHz with correspondingly shrinking budgets. Metrics are angular and energy resolution, gamma efficiency at fixed proton rejection, differential sensitivity, budget-violation rate, and median added latency.
The first decisive check has two parts. On stationary nights, all adaptive variants should tie the static cascade. On drifting nights, the full method should recover a measurably larger share of the all-CNN gain. If it does not, the time-varying framing is unnecessary.
The second decisive check targets premature convergence directly. We inject a sky-background rise that makes low-charge events newly valuable, then track the fraction of such events escalated over time with and without audits. Without audits, the escalation rate in that region should stay near zero long after the shift. With audits, it should recover within a predictable number of audit samples. The audit-cost sweep then shows the price of this insurance on drift-free nights.
If LST collaboration access to the 14-run Markarian 421 dataset can be obtained, a final check replays those runs through each router at matched budgets. We will report detection significance and see where each method lands between the reported 15.3σ and 31.2σ. Disagreement with the deep network is not the same as being correct, especially on real data where the network itself suffers simulation-to-real shift. For that reason, significance on a known source, rather than agreement with the deep network, is the final arbiter.
References
- A 1.8 m Class Pathfinder Raman LIDAR for the Northern Site of the Cherenkov Telescope Array Observatory—Technical Design
- Simultaneously Achieving Sublinear Regret and Constraint Violations for Online Convex Optimization with Time-varying Constraints
- Online Convex Optimization with Perturbed Constraints: Optimal Rates against Stronger Benchmarks
- Optimal sensor scheduling and remote estimation over an additive noise channel
- Remote estimation over a packet-drop channel with Markovian state
- Event-Triggered Time-Varying Bayesian Optimization
- On the Provable Suboptimality of Momentum SGD in Nonstationary Stochastic Optimization
- The Real Time Analysis framework of the Cherenkov Telescope Array's Large-Sized Telescope
- Deep-learning-driven event reconstruction applied to simulated data from a single Large-Sized Telescope of CTA
- Remote Estimation of Markov Processes over Costly Channels: On Implicit Information Benefits
- Divergence-Guided Particle Swarm Optimization