Searches for new particles at colliders increasingly use a machine-learned background template. A generative model learns what ordinary collisions look like just outside a mass window (the "sidebands"), predicts what they should look like inside it, and a classifier then keeps only the events that look least like that prediction. When the classifier is tuned to discard 99.9% or 99.99% of background, the analysis depends almost entirely on how well the template describes the extreme tail. That tail is exactly where generative models are least reliable and where their stated uncertainty is least trustworthy (Accurate and robust methods for direct background estimation in resonant anomaly detection).
Weather and fluid forecasting faced the same problem decades ago. Filters that blend a model forecast with sparse observations become overconfident and drift away from the data. The standard fix is variance inflation, which means deliberately enlarging the forecast's error estimate so the filter keeps listening to observations. We propose to carry two pieces of that toolkit into background estimation. The first is setting the inflation factor from innovations, the gaps between predicted and observed counts in regions where the truth is known. The second is capping the inflated variance with a physically motivated ceiling. The claim is that the template's error grows with selection tightness in a way that can be measured on held-out sideband windows. If so, inflating by that measured amount, with the cap in place, gives correctly calibrated background uncertainties at background efficiency $\epsilon_B = 10^{-4}$, and costs less discovery significance than a single global systematic.
Core Claim and Mechanism
In data-assimilation terms, the signal region plays the role of an unobserved variable. Gottwald, Mitchell and Reich treat such variables as "pseudo-observables" whose filter variance can drift (Controlling Overestimation of Error Covariance in Ensemble Kalman Filters with Sparse Observations: A Variance-Limiting Kalman Filter). The normalizing flow's interpolation into the signal region is the forecast, and the sideband events are the observations.
Two results from that literature motivate the transfer. Kelly, Law and Stuart show that inflation makes filtering contractive: it stops past errors from compounding and keeps the ensemble from collapsing (Well-posedness and accuracy of the ensemble Kalman filter in discrete and continuous time). Law and Stuart show the cost: inflated or approximate filters track the mean well, but their covariance no longer matches the true posterior (Evaluating Data Assimilation Algorithms).
The second result is a warning for physics. Discovery claims rest on the uncertainty, so inflation cannot be ad hoc, unlike the post-hoc variance correction in the exponential-family variational Kalman filter (Real-time variational method for learning neural trajectory and its dynamics). It has to be fixed by a consistency condition that the data can check.
The condition we use is that normalized innovations should have unit variance. Slide a pseudo-signal window $w$ across the sidebands. For each window, refit the flow while excluding $w$ and apply the classifier cut at efficiency $\epsilon$. Then compare the observed count $N_w$ with the template prediction $\hat N_w$, whose own stated variance is $\hat\sigma_w^2$ (statistical plus ensemble spread). The inflation factor is
$$ \lambda(\epsilon) = \frac{1}{|W|}\sum_{w \in W} \frac{(N_w - \hat N_w)^2}{\hat N_w + \hat\sigma_w^2}, $$
where $W$ is the set of validation windows and the denominator is the variance the template claims for that window. If the template were honest, $\lambda$ would be about 1. In the actual signal region, the background uncertainty is replaced by $\lambda(\epsilon)\,\hat\sigma^2$, with $\lambda$ estimated separately at each cut tightness. Any systematic offset in the residuals, if present, is corrected as a bias in the same pass.
The cap is the analogue of the climatological bound in the Variance-Limiting Kalman Filter, which stops an unobserved variable's variance from exceeding its known climatic variance. Here, the inflated template variance may never exceed the variance of a template-free, sideband-only estimate of the same count. That estimate is weaker but honest. The cap keeps inflation from swallowing all of the sensitivity the machine-learned template provides, which is precisely the tension between stability and significance this proposal sets out to resolve.
Research Program
We will build the procedure on top of the CATHODE pipeline: flows learn the sidebands, sample synthetic background into the signal region, and boosted-decision-tree ensembles classify (Isolating Unisolated Upsilons with Anomaly Detection in CMS Open Data). We will also use the cut-and-count direct background estimate of Das et al., which already avoids parametric fits. Our change sits only at the uncertainty layer, so any gain can be attributed to it.
The analysis has three parts. The first is mapping $\lambda(\epsilon)$ across $\epsilon_B$ from $10^{-1}$ down to $10^{-4}$, which tests whether tail miscalibration grows smoothly enough to be measured on sidebands. The second is testing whether $\lambda$ measured in sidebands transfers to the signal region. This requires interpolation error near the window to resemble error at the sideband edges, which is the main assumption and the most likely way the method fails. The third is quantifying how often the cap binds, since that shows where the template is genuinely uninformative.
Baselines
The naive reference is the template's uninflated uncertainty, meaning Poisson counts plus ensemble spread. This is what an analysis gets with no extra effort, and we expect it to undercover in the tails.
The strongest existing method is the direct background estimate of Accurate and robust methods for direct background estimation in resonant anomaly detection, which uses a systematic shift $\delta_{\text{sys}}$ with uncertainty $\sigma_{\text{sys}}$. That paper reports over 5σ discovery significance for both CWoLa and CATHODE at $\epsilon_B = 10^{-3}$ and $10^{-4}$ on the LHC Olympics R&D data, 8σ for an idealized anomaly detector, and low significance on background-only validation. It is the right competitor because it addresses exactly this miscalibration and is already used in practice.
We will rerun it on identical flows, classifiers, training seeds and signal injections, giving it the same number of validation windows to tune $\delta_{\text{sys}}$. Beating it means two things together. First, pull distributions on background-only data must be at least as well calibrated, with coverage of the 68% and 95% intervals within statistical error at $\epsilon_B = 10^{-4}$. Second, the median expected significance at fixed injected signal must be higher.
Three ablations isolate the ingredients. A constant inflation factor, chosen to give the same coverage, tests whether making $\lambda$ depend on $\epsilon$ matters. Inflation without the cap tests what the variance limit buys. The cap without inflation tests whether the bound alone suffices.
Why not just…?
Why not just use a flow ensemble or bootstrap for the uncertainty? Resampling captures the template's variance but not its shared bias. If every ensemble member misses the tail in the same direction, the spread stays small while the error stays large. This is the regime where Das et al. needed a shift parameter at all. Ensemble spread is already inside our $\hat\sigma_w^2$, so a measured $\lambda$ well above 1 is direct evidence that the ensemble alone undercovers. If $\lambda$ comes out near 1, the alternative wins and we will report that.
Why not just quote one large conservative systematic? That works for coverage, but it spends significance at loose cuts where the template is accurate. The constant-inflation ablation is exactly this alternative tuned to equal coverage, so the experiment measures what the $\epsilon$-dependence buys. This is the real threat. If $\lambda$ turns out flat in $\epsilon$, the proposal collapses to it.
Why not just fit a smooth curve to the selected mass spectrum? Das et al. show that parametric bump-hunt fits become unstable at $\epsilon_B = 10^{-3}$ and $10^{-4}$, because too few events survive. They also cannot tell a real resonance from a bump sculpted by the classifier. We include such a fit only as a reference point.
Proposed Experiments
The primary testbed is the LHC Olympics 2020 suite (The LHC Olympics 2020 a community challenge for anomaly detection in high energy physics). On the R&D dataset we inject signal at a range of strengths, including zero, over many seeds. We measure pull coverage, false-discovery rate, and significance at each $\epsilon_B$.
Black Box 2, which is background-only, is the decisive check against false discovery. Black Box 1 tests whether the method recovers a 0.08% signal; the conditional-density approach estimated 1000 ± 200 events against 834 true (Unsupervised in-distribution anomaly detection of new physics through conditional density estimation).
For real data, we reanalyze the anti-isolated Υ search in CMS Open Data, where CATHODE plus boosted decision trees raised significance from 1.6σ to 6.4σ. We will check whether calibrated inflation preserves that significance, and whether the same-sign dimuon control remains null.
Two outcomes would decide the idea. The first is whether $\lambda(\epsilon)$ measured on sidebands predicts signal-region residuals on Black Box 2. The second is whether, at equal coverage, the $\epsilon$-dependent, capped inflation beats both $\delta_{\text{sys}}$ and constant inflation in significance at $\epsilon_B = 10^{-4}$.
References
- Accurate and robust methods for direct background estimation in resonant anomaly detection
- Controlling Overestimation of Error Covariance in Ensemble Kalman Filters with Sparse Observations: A Variance-Limiting Kalman Filter
- Well-posedness and accuracy of the ensemble Kalman filter in discrete and continuous time
- Evaluating Data Assimilation Algorithms
- Real-time variational method for learning neural trajectory and its dynamics
- Isolating Unisolated Upsilons with Anomaly Detection in CMS Open Data
- The LHC Olympics 2020 a community challenge for anomaly detection in high energy physics
- Unsupervised in-distribution anomaly detection of new physics through conditional density estimation