Noise-Conditioned p-Values for Online Discovery Control in Seismic Streams

Deep-learning detectors now find earthquakes and marsquakes far below the level that routine monitoring can see. One workflow grew a five-year catalog in Hainan from 5 events to 977. Another uncovered 39 new marsquakes plus 40 "candidates" in NASA InSight data. None of these pipelines can say how many of their faint detections are false. Each one thresholds a network score over millions of overlapping time windows. The background noise changes constantly, with wind on Mars, industrial activity on Earth, and the time of day. A fixed score threshold therefore means a very different false-alarm rate at 3 a.m. than at noon. Particle physics faced the same problem long ago and adopted strict significance tiers and look-elsewhere corrections. It also learned, painfully, that convenient asymptotic significance formulas overstate weak signals.

We propose to convert any low-signal-to-noise detector's raw score into a p-value calibrated against noise recorded under similar conditions. These p-values then feed an online false discovery rate procedure, which decides in real time which windows to declare as events while bounding the expected fraction of false declarations. The new ingredient is the conditioning. The reference distribution for "what pure noise scores look like" is drawn from noise-only windows matched on a predicted noise state, such as an atmospheric noise forecast or a band-power estimate. It is not a single global pool. Our claim is that this conditioning is what keeps the error guarantee valid in non-stationary noise. Neither a fixed threshold nor a context-free calibration can do this. The conditioning should also recover detections that a global threshold suppresses during quiet periods.

Where the two directions meet, and what is missing

The low-SNR direction concerns signals buried below the noise floor, at a signal-to-noise ratio (SNR) below 0 dB. Detection there relies on structure that noise lacks. Seismic-Net adapts a densely connected CNN to detect CO₂-leakage microseisms. The Hainan workflow chains a U-net phase picker, event association, template matching and a learned denoiser. Stott et al. predict wind- and pressure-induced seismic noise on Mars from weather data, subtract it, and detect events with a threshold that varies with predicted noise variance. These papers motivate the proposal: they show large gains in raw sensitivity. They also expose its gap. Hainan reports false triggers from impulsive cultural noise that required ad-hoc threshold tuning. The InSight study evaluates millions of windows with a semi-empirical threshold, leaves its 40 candidates without any error statement, and fails to model a 2.4 Hz resonance.

The significance direction asks how to report results that fall short of discovery. Three papers shape the design, each in a different role. Neutrino-4 anomaly: Oscillations or fluctuations?, with details in Neutrino-4 anomaly: Oscillations or fluctuations? Neutrino-4 anomaly: Oscillations or fluctuations?, re-analyzes a claimed oscillation signal. Its claimed 3.2σ falls to about 2.7σ once detector resolution is modeled under the asymptotic Wilks approximation, and to about 2.2σ under direct Monte Carlo. This constrains our design. Significance of weak signals must be calibrated empirically against realistic null data, never read off a convenient distribution. Contextual Online False Discovery Rate Control supplies the method. Its CwLORD++ rules let side information steer how much significance budget each incoming test receives while keeping FDR controlled. Those rules were evaluated only on static genomics and clinical tables with precomputed, roughly independent p-values. Das et al. are the closest existing bridge. They compute Bayesian evidence for "event" versus "noise only" at −1.6455 dB with Gaussian-process emulators inside nested sampling, and obtain posterior odds near 4168. Their setting is offline, synthetic and Gaussian. It is expensive enough that the weakest event needed more sampler live points, and it gives no frequentist error rate over a stream.

The obvious combination would be to "run CwLORD++ on detector outputs." That idea is not in the file. It is also not sufficient on its own. Online FDR rules are only as valid as their p-values. A deep detector's score has no p-value until someone specifies the noise it should be compared against. In non-stationary, colored, spiky noise that specification is the hard part. Our contribution sits there.

Core Claim

Let $s_t$ be the detector score for window $t$, and let $z_t$ be a predicted noise state for that window. For InSight, $z_t$ is the weather-based noise forecast. For terrestrial stations, it is band-limited power over the preceding minutes plus hour-of-day. We keep a calibration pool of past windows certified as noise-only, either by catalog absence or by time-shifting multi-station data so that no real moveout remains. From this pool we take the $n$ windows whose states are nearest to $z_t$. The noise-conditioned p-value is

$$ p_t = \frac{1 + \#\{i \le n : s_i \ge s_t\}}{n+1}, $$

where $s_i$ are the detector scores of those matched noise windows. This value is valid, meaning that under pure noise it is no smaller than uniform, whenever noise windows sharing a noise state are exchangeable. That condition is a local, checkable assumption, and it replaces the global stationarity that fixed thresholds silently require. The p-values enter a LORD++-style online procedure. The per-test level $\alpha_t$ is allowed to depend on context through CwLORD++, for example array coherence or distance from known active faults. To limit dependence between overlapping windows, hypotheses are defined on declustered, event-length blocks.

We predict two things. Under realistic non-stationary noise, the realized FDR of this pipeline will stay near nominal, while a single global threshold set to the same average false-alarm rate will exceed it in noisy periods. At matched realized FDR, the conditioned pipeline will also detect more weak events, because quiet periods are no longer held to the noisy periods' threshold.

Research Program

The work has three parts. The first is building the calibration machinery. This covers certifying noise-only windows, choosing the noise-state features and neighborhood size $n$ (a bias–variance tradeoff, since small $n$ gives coarse p-values and large $n$ mixes states), and diagnosing calibration. For diagnosis we plot held-out noise p-value histograms per noise-state stratum. This is our direct analog of the Monte Carlo check that overturned the Neutrino-4 claim.

The second part addresses dependence. Overlapping windows and colored noise make successive p-values positively correlated, and the guarantee for standard online FDR rules can then weaken. We will compare block-level hypotheses at several block lengths. We will report realized FDR empirically, and we will not claim a theorem we do not have.

The third part is reporting. Each declared event receives a calibrated p-value and an online FDR level at which it survives. This gives seismology an analog of physics' "evidence" and "hint" tiers that rests on measured error rates. We will apply it to the 40 InSight candidates.

Proposed Experiments

Ground truth is the central obstacle, because the gray zone is defined by our not knowing. We therefore use injection. Real cataloged waveforms are scaled to a range of SNRs, from comfortably above the noise down to −10 dB. They are inserted at known times into real noise-only recordings. Precedent for this is the Oklahoma study's augmentation of real events with scaled signals for magnitudes $3.0 > M_L \ge 0.5$ (Zhang et al.). Injection rates are set low, so that true events are rare as they are in practice.

The test beds come from the file. The Chimayó CO₂-leakage dataset, with 23 training days and 32 test days, has a published detector to reuse and a published operating point. Northeastern Hainan offers five years of five-station 100 Hz data with strong diurnal cultural noise. InSight SEIS with the TWINS weather data has the most extreme weather-driven non-stationarity, and the Stott noise predictor supplies $z_t$ almost directly.

Detectors are held fixed across all statistical procedures: Seismic-Net on Chimayó, a PhaseNet-style picker on Hainan, and the InSight variable-threshold detector's score on Mars. The comparison therefore isolates the decision layer. Metrics are realized FDR, stratified by noise-state decile; true detections at matched realized FDR, as a function of injected SNR; and latency per decision.

Baselines

The naive reference is the detector's own operating threshold, applied uniformly. On Chimayó this is the point at which Seismic-Net reports precision 0.889 and recall 0.923, with 24 of 26 test events detected (Seismic-Net). We rerun this threshold on the same injected streams and measure its realized FDR per noise stratum. The published number covers only 26 events and does not show behavior at injected SNRs below the test set.

The strongest practical competitor is a global false-alarm-rate threshold. The threshold is set from the full pool of noise-only scores so that, say, one false trigger per day is expected, and it is then corrected for the number of windows scanned. This is what careful monitoring groups would actually do, and it is a look-elsewhere-corrected version of the naive threshold. It uses the same calibration pool and the same detector. Beating it means one of two things: lower worst-stratum realized FDR at equal total detections, or more true detections at equal worst-stratum FDR, specifically among injections below 0 dB.

For InSight, the strongest existing method is the Stott variable threshold tied to predicted noise variance. No FDR or false-alarm figure is reported on Lacuna for that method. The file gives only 39 new events, 40 candidates, and detectivity of roughly 100%, 45–50% and 5–20% for large, median and small events. We reproduce that threshold on injected Mars noise and compare detectivity at matched realized FDR.

For the statistical layer, the reference is plain LORD++ on context-free conformal p-values. The context-free version calibrates against the global noise pool and is otherwise identical to ours. This is the key ablation: it isolates noise conditioning from online FDR itself. A second ablation adds CwLORD++ context to the conditioned p-values, to measure whether alpha allocation helps beyond calibration. The only reported figure for CwLORD++ is from genomics, a roughly 10% increase in discoveries over LORD++ on Airway RNA-Seq (Contextual Online FDR Control). No reported result was found on Lacuna for any seismic stream. Offline Benjamini–Hochberg on the conditioned p-values serves as an upper reference for power, since it sees the whole stream at once. All procedures share the same detector, calibration windows, nominal level $q \in \{0.05, 0.1, 0.2\}$, and tuning budget. Neighborhood size is chosen on a separate validation month.

Why not just…?

Why not just set a global false-alarm-rate threshold from noise-only data? This is the real threat. If noise in a deployment is close to stationary, a global threshold and our method coincide, and ours adds only overhead. The question is settled by the stratified FDR measurement. The Stott study shows that detectability on Mars swings strongly with weather, and Hainan's false triggers came from intermittent spikes. Both suggest that a global threshold over-fires in loud strata and under-fires in quiet ones. If Chimayó shows no stratum effect, we will report that our method is unnecessary there.

Why not just denoise first and then threshold? Noise subtraction, as in Stott et al., raises SNR but gives no error rate. It also leaves unmodeled residue such as the 2.4 Hz resonance, which a fixed threshold will treat as signal. Denoising is complementary: we run the conditioned calibration on denoised scores too. The noise-only pool then absorbs whatever residue the denoiser leaves.

Why not just compute Bayesian evidence per candidate, as Das et al. do? It requires a forward model and many likelihood evaluations per window. The paper needed extra live points just for its weakest synthetic event. It also reports posterior odds rather than a stream-level error rate. We use it as a motivating comparison, not a scalable baseline.

Why not just run Benjamini–Hochberg offline over a month of triggers? Retrospective catalogs can do exactly that, and we include it as the power ceiling. Traffic-light systems for induced seismicity need decisions as events arrive, which is the setting the Oklahoma study targets with its 0.01-second inference. Reporting how close the online rule comes to the offline ceiling tells practitioners what immediacy costs.

Decisive Checks

The proposal succeeds if three things hold. Conditioned p-values must be approximately uniform on held-out noise in every noise-state decile, while context-free ones are not. The full pipeline must keep realized FDR at or below $q$ in the worst decile where the global threshold exceeds it. And at matched worst-decile FDR, the pipeline must recover more injected events below 0 dB on InSight and Hainan. The proposal fails if conditioned and global calibration give the same stratified FDR. That would show that score non-stationarity is not the bottleneck. As a final scientific product, we will assign p-values to the 40 InSight candidates and report which survive online control at $q = 0.1$. This turns an unqualified candidate list into a graded statement of evidence.