Upper Limits That Know Their Subtraction Floor

Searches for dark matter in gamma-ray data usually work by elimination. Analysts model and remove everything they know is there, such as bright point sources, diffuse gas emission and cosmic-ray foregrounds. They then ask how large a dark-matter signal could hide in what remains without being noticed. The answer is an exclusion limit: a boundary in the space of particle mass and interaction strength beyond which the data say "no" at 95% confidence. The step this proposal targets is the removal itself. Wireless engineers have studied that step closely under the name successive interference cancellation (SIC): decode the loudest signal, rebuild it, subtract it, then decode the weaker signals underneath. Their central lesson is that subtraction is never perfect. The leftover error grows with the brightness of what was removed, so it forms a floor that more signal power, or in astronomy more exposure, cannot push through (Outage Performance of A Unified Non-Orthogonal Multiple Access Framework).

The claim here is that gamma-ray exclusion limits near bright, subtracted structure carry the same floor, and that current likelihoods ignore it. Those likelihoods treat what remains after subtraction as pure Poisson counting noise. Near bright sources, that should make the limits too tight, and it should make the stated sensitivity improve with exposure when in fact it has stopped improving. We propose to add a calibrated, brightness-dependent residual term to the standard likelihood, and to propagate catalog misidentification through it as a "wrong decode" error. The test is whether the resulting limits have correct statistical coverage in the Fermi Large Area Telescope (Fermi-LAT) Galactic-plane region, where gas clumps are known to masquerade as point sources.

Core Claim

Consider pixel $i$ of a counts map. Let $b_i$ be the expected counts from the smooth background, $s_i$ the expected counts from discretely subtracted components (catalog point sources and gas clumps), and $\beta$ the fraction of subtracted counts that survives as structured, unmodeled residual. The residual comes from point-spread-function errors, spectral mismodeling and position errors. In the SIC analysis, the leftover interference power is $\beta$ times the strong signal's power, which caps the signal-to-interference ratio at $P_{\text{weak}}/(\beta P_{\text{strong}})$ (Outage Performance of A Unified Non-Orthogonal Multiple Access Framework). The astrophysical analogue is a per-pixel nuisance with standard deviation $\beta s_i$. With exposure time $T$, both $b_i$ and $s_i$ scale linearly in $T$. The smallest excludable signal rate therefore scales as

$$ \sigma_{\text{lim}}(T) \;\propto\; \frac{\sqrt{b_i T + \beta^2 s_i^2 T^2}}{T}, $$

The first term under the root is Poisson counting noise and the second is the residual nuisance. The limit falls as $1/\sqrt{T}$ until $T \approx b_i/(\beta^2 s_i^2)$, then flattens at a value proportional to $\beta s_i/T$. That is the exposure-domain counterpart of the "diversity drops to zero" error floor.

The model makes two predictions that a flat background systematic does not. The floor appears only in pixels near subtracted structure. It is also spatially correlated over the instrument's point spread function, unlike white noise. If either prediction fails in real data, the claim is wrong.

The second ingredient comes from SIC error propagation. In wireless SIC, subtracting a wrongly decoded signal injects extra noise and corrupts every later stage (Amanat Kafizov et al.). In gamma-ray analysis, the corresponding mistake is fitting a gas fluctuation as a point source. Karwin et al. found 23 such point-like gas features in the Mopra region. Their modified template was detected at about 48σ in simulations, and 35% of the new features overlap "unassociated" 4FGL-DR2 sources. We treat each such catalog entry as a decode with a misidentification probability. We then marginalize the limit over catalog realizations rather than conditioning on one fixed catalog.

Relation to Prior Work

The cancellation machinery comes from communications. Sequential subtract-and-decode with imperfect cancellation and its error floor is worked out for power-domain and code-domain multiplexing in Outage Performance of A Unified Non-Orthogonal Multiple Access Framework. Error propagation is studied by Amanat Kafizov et al.. We borrow the residual model and the wrong-decode model but apply them to Poisson counts maps and to statistical limits rather than to outage probability.

Four papers apply cancellation in sensing, and they set the closest precedent. Multiple fluctuating targets track-before-detect using multi-Bernoulli filter in radar sensor subtracts the brightest radar target to expose weak ones. Concurrent Downlink and Uplink Joint Communication and Sensing for 6G Networks subtracts a loud communication signal to recover faint radar echoes, improving range error by more than 10 dB. NOMA Inspired Interference Cancellation for Integrated Sensing and Communication treats radar beams as virtual users to be cancelled. Revealing the Impact of SIC in NOMA-ISAC compares cancellation orders, though only under perfect cancellation. All four aim at detection or point estimation in Gaussian noise. None builds exclusion limits or asks whether a stated confidence level is honest after imperfect subtraction, which is the question this proposal adds.

On the astrophysics side, two papers motivate the work by showing that sensitivity depends heavily on how foregrounds and morphology are modeled. Constraints on Ultraheavy Dark Matter Properties from Dwarf Spheroidal Galaxies with LHAASO Observations reports that extended-source modeling loosens limits by a factor of 2 to 6. Combined search in dwarf spheroidal galaxies for branon dark matter annihilation signatures with the MAGIC telescopes shows that limits shift by several-fold between two sets of dark-matter density estimates for the target galaxies, the line-of-sight density integrals called $J$-factors.

Karwin et al. both motivates and constrains the proposal. It shows that catalogs contain mis-decodes, and its improved gas template is the strongest existing remedy, which our method must beat or complement.

Research Program

The work has four parts.

Calibrating the residual fraction. We estimate $\beta$ empirically in control regions of the Fermi-LAT 12-year P8R3_CLEAN data away from the target strip. We fit isolated 4FGL-DR2 sources with fermipy, stack the residuals as a function of fitted source counts and energy, and fit $\beta(E)$ together with a correlation kernel tied to the point spread function. The calibration must be done on held-out sky, so that $\beta$ is not tuned to the region where limits are set.

Adding the nuisance to the likelihood. The term is added as a Gaussian-approximated, spatially correlated nuisance on the subtracted components inside the standard binned Poisson likelihood. We profile over it. The same structure can later be carried to the joint dwarf-galaxy likelihoods used by MAGIC and LHAASO.

Modeling misidentification. We assign each unassociated catalog source overlapping Mopra $^{13}$CO structure a probability of being gas rather than a true source. The probability is estimated from the cross-match rate in Karwin et al.. We then compute limits marginalized over sampled catalog realizations. In each realization, sources labeled as gas are moved into the gas template rather than deleted.

Ordering check. Following the ordering analysis in Revealing the Impact of SIC in NOMA-ISAC, we refit with the dark-matter template fitted before versus after the point sources. The size of the shift in the limit is a cheap per-region warning of leakage between the two.

Baselines

The naive reference is the standard pipeline. It runs a fermipy joint fit with GALPROP-based diffuse templates, keeps 4FGL-DR2 source positions fixed, lets source normalizations float, and uses a pure Poisson likelihood. That is "treat what remains as noise," the scheme Nikbakht et al. found inadequate in communications when high reliability is needed.

The strongest existing method for this region is the Karwin et al. modified gas template added to the same fit. It directly removes much of the gas-as-point-source contamination. The second strong competitor is a flat fractional systematic on all background components, a common practice in the field. We give it the same calibration budget, fitting its single fraction on the same control regions.

A masking baseline excludes pixels within a radius of bright sources, with the radius tuned on control regions.

Two ablations isolate the new ingredients. One sets $\beta=0$ while keeping the catalog marginalization. The other replaces the brightness-dependent residual with a spatially uniform one of matched total variance.

All methods use identical data cuts, energy binning, templates, region of interest and number of free parameters in the fit. Each gets the same calibration sky.

No Lacuna paper reports limit coverage near subtracted sources. The closest reported figures concern contamination rather than coverage: the 48σ template detection, the 23 gas features and the 35% catalog overlap from Karwin et al. Beating the baselines means two things. First, the 95% upper limits must achieve empirical coverage within a few percent of 95% in injection tests, in the pixels where the baselines undercover. Second, the limits must lose less depth than masking does in the pixels where masking is used. Matching the Karwin template's coverage while also predicting where the exposure floor appears would count as a complementary success, not a failure.

Why not just…?

Why not just do a simultaneous joint profile likelihood fit? The naive baseline already is one. A joint fit is not wrong because it fits components simultaneously. It is wrong because its noise model has no term for structured residuals from misspecified templates. A better optimizer cannot repair that. Karwin et al. show that joint fits still assign gas emission to point sources. The injection tests will show whether joint-fit limits undercover near bright structure.

Why not just add a flat background systematic? This is the most serious threat. A flat systematic loosens limits everywhere, so its single fraction must trade off overcoverage far from bright sources against undercoverage near them. Our model predicts that this compromise fails in a specific spatial pattern. If a flat systematic achieves equal coverage at equal limit depth across the brightness bins, the brightness-dependent structure is unnecessary, and we will report that outcome.

Why not just mask bright sources? Masking is the orthogonal-resource answer. It removes the floor by discarding sky, and the Galactic plane is where that sky matters most. We will measure how much limit depth is lost per unit of coverage gained. The proposal is useful only if keeping the pixels while widening their uncertainty beats throwing them away.

Why not just use better templates, such as Karwin's? Better templates lower $\beta$ but do not remove it. The decisive check is whether the exposure floor persists after the improved template is included. If it vanishes, better templates suffice and our contribution reduces to a diagnostic.

Proposed Experiments

Injection–recovery coverage. We inject faint dark-matter-like templates into the Mopra strip region of the real 12-year data. The templates include NFW-profile annihilation and a compact extended blob, at fluxes spanning detectable to sub-threshold, and are placed at controlled distances from bright 4FGL sources and from Mopra $^{13}$CO clumps. For every method, we measure the fraction of injections falsely excluded at 95% confidence, binned by local subtracted counts $s_i$.

Floor test. We split the data into nested exposures of 1, 2, 4, 8 and 12 years. We then check whether the limits stop improving near bright structure at the exposure the calibrated $\beta$ predicts, while still improving as $1/\sqrt{T}$ in clean pixels. This prediction is falsifiable and does not depend on the injections.

Catalog-swap test. We recompute limits after reclassifying the 23 Karwin gas features. This quantifies the wrong-decode cascade directly and checks that the marginalized limits bracket both choices.

Transfer test. As a secondary check, we apply the calibrated nuisance to Fermi-LAT dwarf-galaxy fields. We expect it to change little there, because those regions contain little bright subtracted structure. A large shift would mean the nuisance is absorbing something other than subtraction residuals.

For context only, published reference limits include MAGIC's $\langle\sigma v\rangle \sim 1.9\times10^{-24}$ cm³ s⁻¹ at 1.5 TeV (Abe et al.) and LHAASO's roughly $10^{-23}$ cm³ s⁻¹ at 100 TeV (LHAASO Collaboration). They come from different instruments and are not direct comparisons.