How Much Do Stacked Background Cuts Really Reject? Dependence-Robust Leakage Bounds for Rare-Event Searches

Experiments that hunt for extremely rare events, such as dark matter scatters or neutrinoless double-beta decay, rarely trust a single test to remove background. They stack several: a timing-shape test, a track-shape test, a test of where the event sits in the detector. Each test lets through a measured small fraction of background events, called its "leakage". The number that matters for a discovery claim is how much background survives all of the tests together. That joint survival rate is often too small to measure directly, so it is tempting to multiply the individual leakages. Multiplying is only correct if the tests fail independently on background events. In practice the tests frequently share a physical cause. An event near a detector surface can fool both a charge-collection test and a pulse-amplitude test for the same reason. When failures are correlated, the product of leakages is optimistic. It can be optimistic by orders of magnitude, and nothing in the product itself warns you.

This is the same failure the information-fusion literature identified decades ago. Combining Bodies of Dependent Information showed that multiplying evidence from sources with a shared bias raises that bias to a power and produces false confidence. Weak-supervision methods have since found ways to aggregate dependent sources safely. We propose to carry one of those ideas into rare-event physics. The idea is to compute the worst case over every joint distribution consistent with what was actually measured, in the spirit of Adversarial Multi Class Learning under Weak Supervision with Performance Guarantees. The output is a statistically valid upper bound on joint background leakage that requires no independence assumption. We pair it with a diagnostic that reports how much of a published rejection factor depends on assuming independence.

Core Claim

The joint leakage of $m$ stacked cuts can be bounded from above, without any independence assumption, using only quantities that are cheap to measure. These are the leakage of each cut at its tight working threshold, measured on that cut's own large calibration sample, and the joint pass rates of pairs and triples of cuts on a shared but smaller sample. The second set can be measured at looser thresholds where statistics allow. The key observation is that cut regions are nested: an event passing a tight threshold also passes every looser one. Joint rates measured at loose thresholds are therefore valid upper bounds on the unmeasurable tight-threshold joint rates. Let $p$ be a probability vector over the $L^m$ cells formed by $L$ threshold levels on each of the $m$ cuts. Let $a_k^\top p$ be the probability of the $k$-th measured event, for example "cut 1 tight and cut 3 loose both pass". Let $[\ell_k, u_k]$ be its confidence interval. The bound is

$$ \hat{U} \;=\; \max_{p \ge 0,\ \mathbf{1}^\top p = 1} \; p_{\text{all tight}} \quad \text{subject to} \quad \ell_k \le a_k^\top p \le u_k \ \text{ for all measured } k . $$

The bound $\hat{U}$ is a linear program, a constrained optimization with a linear objective and linear constraints. It holds with the joint confidence level of the intervals. When the only constraints are the individual marginals, it reduces to the classical Fréchet bound, which is the smallest marginal leakage. Every added pairwise or loose-threshold measurement can only tighten it. The gap between $\hat{U}$ and the independence product $\prod_i \epsilon_i$, where $\epsilon_i$ is the leakage of cut $i$ alone, measures how far a claimed rejection factor rests on assumption rather than measurement. We expect that for realistic, physically coupled discriminators this gap is large at leakages near $10^{-6}$ to $10^{-7}$. If it is not, the independence shortcut is vindicated, and that is also worth knowing.

How this differs from prior work

The closest work joining weak supervision and particle physics is Classification without labels: learning from mixed samples in high energy physics. It trains a classifier on two mixed samples and proves the optimal mixture classifier is the optimal signal-versus-background classifier. That work is about learning a decision rule. It says nothing about certifying the background leakage of a stack of rules, and it assumes the class-conditional distributions are the same in both samples. The multi-background anomaly detection papers, Robust Anomaly Detection for Particle Physics Using Multi-Background Representation Learning and Resonant anomaly detection with multiple reference datasets, target outlier discovery rather than rate estimation. Learning with Weak Supervision from Physics and Data-Driven Constraints adds physical constraints to label learning but does not handle correlated errors among discriminators.

On the weak-supervision side, three papers supply our tools while targeting a different object, namely training labels. Adversarial Multi Class Learning under Weak Supervision with Performance Guarantees chooses the worst-case labeling inside a feasible set defined by measured source error rates. We take this worst-case-over-a-feasible-set construction and apply it to the joint distribution of cut outcomes instead of to labels. Constrained labeling for weakly supervised learning shows that writing weak signals as linear constraints avoids double-counting redundant sources. We share the linear-constraint view and replace its least-squares solve with an extremal bound. Comparing the Value of Labeled and Unlabeled Data in Method-of-Moments Latent Variable Estimation proves that unmodeled dependencies leave a bias that more data cannot remove. That result motivates the proposal: more calibration events at the marginal level will never fix a product estimate. Its median-over-subsets correction suggests our point estimator, described below.

The physics papers supply the setting. In Topological background discrimination in the PandaX-III neutrinoless double beta decay experiment, four track-shape cuts combine to roughly seven orders of magnitude of suppression. The four cuts are track length, blob charge (energy deposited at a track end), end-track twist, and energy ratio, and all of them derive from the same electron energy loss and scattering. In $\alpha$-event characterization and rejection in point-contact HPGe detectors, two germanium-detector tests depend on the same internal electric field. These are delayed charge recovery, which reads the slow tail of the waveform, and the amplitude-to-energy ratio. Together they reject surface alphas with 0.2% bulk loss. Both stacks are natural candidates for correlated leakage. We do not claim either paper misestimated its numbers. We claim the field lacks a general tool for checking such numbers when joint statistics run out.

Research Program

The work has three parts. The first is the bound itself. We build the threshold-ladder linear program above, with simultaneous confidence intervals (Clopper–Pearson with a Bonferroni split, and a bootstrap alternative). The program also handles the realistic data layout, in which each cut's tight marginal comes from a separate large calibration source and the joint cells come from a smaller shared sample. We will characterize how the bound tightens as joint measurements are added, and when it collapses to Fréchet.

The second part is a point estimator to sit beside the bound, because experiments need a central value as well as a ceiling. We fit a sparse dependency model over the cut-pass indicators across the threshold ladder. This reuses the sparsity-penalized structure selection of Learning the Structure of Generative Models without Labeled Data, simplified to the supervised case, since sideband data are nearly pure background. We then extrapolate to the all-tight cell. As a robustness check borrowed from the median correction, we compute the joint leakage through every chain-rule factorization, $P(A)P(B\mid A)P(C\mid A,B)$ and its permutations. Each conditional is estimated at the tightest level the data support, and we report the median and the spread. A large spread is itself a warning that the cuts interact.

The third part is a diagnostic. It reports the "independence share" of a claimed rejection factor, $\log(\hat U)/\log(\prod_i \epsilon_i)$. This is the fraction of the claimed rejection, in orders of magnitude, that survives without assuming independence. The same machinery also gives a lower bound on joint signal efficiency from calibration proxies, which matters because signal acceptances are multiplied in the same way.

Baselines

The naive reference is the independence product $\prod_i \epsilon_i$, computed from each cut's marginal leakage on the same calibration samples our method sees. Its error relative to the true joint leakage is the quantity the project exists to measure. The trivially valid reference is the Fréchet bound $\min_i \epsilon_i$, which is always correct and usually useless. Our linear program must be strictly tighter than it, or the extra measurements bought nothing.

The strongest competitor is not a combination rule at all. It is to train one multivariate classifier on all discriminator inputs and apply a single cut to its score. This removes the multiplication step entirely. The tail of that score must then be extrapolated, typically by fitting a parametric form to the score distribution of background calibration events. On the simulated Higgs-to-tau-tau benchmark, the 6-layer network of Searching for Higgs Boson Decay Modes with Deep Learning reports an AUC of 0.803, compared with 0.798 for the best shallow network. We will use a network of that kind as the classifier. For the task we actually evaluate, no reported result was found on Lacuna for leakage estimation accuracy at tiny rates, for any method. That includes the FAIR Universe benchmark, where FAIR Universe HiggsML Uncertainty Dataset and Competition reports no numeric scores in our file.

For fairness, every method sees the same calibration events at each budget. The single-score baseline gets the same tuning budget for its tail fit as our estimator gets for its structure penalty. Comparisons are made at matched signal efficiency. Our method beats the strongest baseline if two conditions hold. First, our point estimate has lower median log-error against true joint leakage at target rates of $10^{-5}$ to $10^{-7}$. Second, our upper bound reaches nominal coverage where the tail-fit interval does not.

Two further references isolate what matters. A Gaussian copula, a model that couples the marginals through a single correlation matrix, tests whether "just model correlation" is enough. The ablations remove the loose-threshold constraints, keeping only tight marginals and tight pairs, and then remove the pairwise constraints. These show which measurements carry the tightening.

Why not just…?

Why not just train one classifier and cut on its score? This is the real threat, and we treat it as the main comparison rather than dismiss it. A single score avoids multiplication but not extrapolation. With $N$ background calibration events, any leakage below about $1/N$ is still unmeasured, and the tail-fit model now carries the assumption the product used to carry. The experiment below decides the question directly. If tail-fit leakage estimates for a learned score are as accurate and as well covered as our bounds, the practical advice becomes "use one score". Our method then reduces to auditing legacy stacked cuts.

Why not just compute the joint leakage in Monte Carlo? Simulation can generate $10^8$ events, but FAIR Universe HiggsML Uncertainty Dataset and Competition and Classification without labels: learning from mixed samples in high energy physics both stress that simulated correlations differ from real detector behavior. Correlations driven by surface effects and field nonuniformities are exactly the details simulation gets wrong. We use simulation only as ground truth for validation. Our shifted-nuisance experiment tests whether dependence fitted at one systematic setting transfers to another.

Why not just fit a correlation matrix or Gaussian copula? It is cheap, but a Gaussian copula has no tail dependence: correlations measured in the bulk vanish in the extreme corners. That is precisely where correlated leakage lives. It is one of our baselines, so the data will show whether its error grows as the target leakage shrinks, as we expect.

Why not just use larger calibration samples for each cut? The method-of-moments result in Comparing the Value of Labeled and Unlabeled Data in Method-of-Moments Latent Variable Estimation is the analogue here. More data on the wrong quantities leaves the dependence bias in place. Larger marginal samples sharpen $\epsilon_i$, but they leave the product's bias untouched. Only joint measurements constrain the joint.

Proposed Experiments

The main testbed is the FAIR Universe HiggsML dataset of 280 million simulated events. It has 28 features and six nuisance parameters, which are adjustable knobs representing calibration and theory uncertainties. Its size lets us compute true joint background leakage down to roughly $10^{-7}$ by brute counting. We build stacks of three to six cuts on physically related high-level features, so that some stacks share mechanisms and others do not. We then hand each estimator subsamples of $10^4$ to $10^6$ events in the realistic separate-marginal, small-joint layout. The 82-million-event Higgs-to-tau-tau benchmark from Searching for Higgs Boson Decay Modes with Deep Learning serves as a replication with different physics. If collaborators can supply them, simulated track-topology events from the REST setup used for PandaX-III would give a direct test on the four coupled topological cuts.

We report four metrics. These are the log-ratio of estimate to truth, the empirical coverage of the 90% upper bound over repeated subsamples, the tightness of the bound relative to truth and relative to Fréchet, and the independence share defined above. Each is reported as a function of target leakage and calibration budget.

The first decisive check tests the premise. On mechanism-sharing stacks, we ask whether the independence product underestimates true leakage by more than a factor of three at $10^{-6}$. If it does not, across both datasets, the motivation weakens and we say so. The second check is whether the linear-program bound reaches at least 90% coverage while staying at least an order of magnitude below Fréchet. Without that margin, the extra measurements buy nothing. The third check is the head-to-head against single-score tail fitting at matched signal efficiency. The fourth is a transfer test: calibrate at nominal nuisance values, evaluate at shifted ones, and check whether coverage of the bound survives while the product and copula estimates drift. Together these establish when stacked rejection claims can be trusted, and give an honest number when they cannot.