Astronomers now hunt for extremely faint, spread-out structures: ultra-diffuse galaxies, which are galaxy-sized but nearly transparent, and tidal tails, which are streams of stars pulled off a galaxy by a neighbor. Each pixel of these objects can carry a signal a tenth of the sky's noise. Detection pipelines and neural detectors for such "low surface brightness" (faint light per unit sky area) structures are multiplying. The number every science case needs, though, is how often a detector finds a real structure of a given faintness in a given survey, and nobody can currently measure it well. The usual answer is to plant synthetic structures into real images and count how many come back, a procedure called injection-recovery. Its accuracy depends on whether the planted objects look like real ones. The alternative, counting recoveries of confirmed real structures, rests on tiny samples. The most dramatic known example, the 60 kpc tail of the galaxy F8D1 at about 32 mag arcsec⁻², was confirmed only by counting individual stars, which is possible only for very nearby galaxies (A Tale of a Tail).
We propose to combine the two. Thousands of injected structures fix the shape of the detection-probability curve as a function of faintness. A small set of 10 to 40 real structures, confirmed in overlapping deeper imaging, estimates only a low-dimensional "realism correction" to that curve, with uncertainty that honestly reflects the small sample. The resulting completeness curves should keep their stated error bars when moved to a new survey, where raw injection curves become overconfident and real-only estimates become too wide to use. Biomarker research faces the same trap: small validation cohorts yield striking but fragile numbers, such as an AUC of 92.5% from 20 patients versus 20 controls in APOC1 as a novel diagnostic biomarker for DN. Faint-structure astronomy has an advantage medicine lacks: a simulator that can generate unlimited labelled positives. We ask how to use it without trusting it blindly.
Core Claim
Let $C(\mu)$ be the probability that a detector recovers a real structure of mean surface brightness $\mu$. Let $C_{\text{inj}}(\mu)$ be the same probability measured on injections. We model the relation as
$$\text{logit}\, C(\mu) = \text{logit}\, C_{\text{inj}}(\mu) + \delta_0 + \delta_1(\mu - \mu_0),$$
where $\delta_0$ and $\delta_1$ are a realism offset and slope, and $\mu_0$ is a reference faintness. The offset and slope are fit by Bayesian logistic regression on the few real structures, with $C_{\text{inj}}$ held fixed from thousands of injections. The prediction is that this two-parameter correction captures most of the gap between synthetic and real objects. If so, 90% intervals on $C(\mu)$ should reach close to nominal coverage with only tens of real examples.
A second claim can be tested separately. If the realism gap mostly reflects the injection model rather than the survey, then $\delta$ fitted on one survey pair should transfer to another. If it does not transfer, that result is itself useful, because it tells surveys they must each collect their own small confirmed set.
The contribution is a validation method, not a new detector. Existing work improves detectors but does not ask how their faint-end completeness is certified. That includes cross-survey domain adaptation, pixel-level instance segmentation, and regularized profile fitting. The closest work, DeepAstroUDA, measures cross-survey accuracy on bright spirals and ellipticals with large labelled source sets (DeepAstroUDA). We target diffuse structures below the sky noise and a regime where real labels number in the tens.
Research Program
The central construction is a set of paired-depth fields. In each pair, a deep image supplies "real" labels for a shallower image of the same sky. We use three pairs.
The first pair is simulated Rubin Observatory LSST 1-year versus 10-year co-adds, from the DeepAstroUDA benchmark. It provides exact truth. The second is SDSS Wide versus Stripe 82 deep co-adds, the depth transition also studied in DeepAstroUDA. The third is HSC UltraDeep COSMOS (DeepDISC) against noise-degraded copies of itself. In each pair, structures visible in the deep image and catalogued by visual inspection plus profile excess become the real positives for the shallow image. From that pool we draw small calibration sets of size 10, 20 and 40. The rest is held out as a large reference against which estimated curves are scored.
Injections span a controlled family of synthetic structures, inserted into the shallow images across a grid of $\mu$ from 25 to 31 mag arcsec⁻²:
- smooth exponential ultra-diffuse galaxies;
- curved tails with varying width and clumpiness;
- structures placed near bright stars and Galactic cirrus, a known contaminant that A Tale of a Tail had to correct with WISE infrared templates.
We deliberately include both a simple injection model and a richer one. This shows how the correction behaves when realism is poor versus good.
We audit several detectors so that the method is not tied to one of them:
- Threshold detection:
photutils-style pixel-threshold detection. - AutoProf profile excess: outskirts-excess detection built on AutoProf. AutoProf penalizes jumps in ellipse shape between radii and averages pixels in radial bands, which let it trace isophotes more than 2 mag arcsec⁻² deeper than
photutils. - Fine-tuned DeepDISC: the transformer instance segmenter of DeepDISC, fine-tuned on injections.
- Gaussian-process classifier: a Gaussian-process classifier on compressed cutouts, following Star–Galaxy Image Separation with Computationally Efficient Gaussian Process Classification. It beat a morphological classifier's 0.9 accuracy with about 80 training examples and reports its own uncertainty.
For each detector and pair, we estimate $C(\mu)$ three ways: injection only, real only, and corrected. We then compare the estimates against the held-out reference. Finally we fit $\delta$ on one pair and apply it to the others to test transfer. F8D1 serves as a single-object stress test in CFHT MegaCam integrated light. We report whether the corrected curve assigns its 32 mag arcsec⁻² tail a detection probability consistent with its absence from integrated-light searches.
Baselines
Do nothing. The naive reference is the practice that dominates: report injection-recovery completeness as if it were real completeness. It is run with the same injections, detectors and shallow images as the corrected method, so the only difference is the absence of the real-data correction. The corrected method beats it if its 90% intervals cover the held-out truth near 90% of the time while the injection-only curve's coverage is substantially lower, especially below 28 mag arcsec⁻².
Real-only estimate. The second baseline uses only the small real set. It is binned or logistic completeness with exact binomial intervals, which is the honest small-sample analogue of the clinical validation studies in the file. It uses the identical calibration draws. The corrected method beats it if it matches its coverage with intervals at least a factor of two narrower at $n = 20$.
Strongest detector-side competitor. The strongest existing method is not a completeness estimator but a detector that claims to generalize across surveys: DeepAstroUDA's clustering and entropy-separation adaptation, which raised unlabelled-target accuracy by 36% to 40% on SDSS-to-DECaLS and reached anomaly AUC of 0.81 to 0.92 (DeepAstroUDA). For faint-structure completeness above 28 mag arcsec⁻² there is no reported result on Lacuna, for this or any detector. We run DeepAstroUDA-style adaptation from the deep to the shallow domain of each pair, with the same unlabelled target images and the same hyperparameter budget as the DeepDISC fine-tune. We then ask two questions. First, does adaptation shrink $\delta$? Second, does its own entropy score predict real misses as well as the corrected curve does?
Self-reported uncertainty. A related baseline uses the Gaussian-process posterior probabilities from the Muyskens-style classifier directly as completeness. This tests whether model-internal uncertainty is already calibrated under shift.
Key ablations. The first ablation replaces the two-parameter correction with an offset-only version, and separately with a flexible spline, fit on the same small sets. This isolates whether the low-dimensional form is what buys coverage at small $n$. The second ablation swaps the simple injection model for the richer one, which shows whether the method's value shrinks as realism improves.
Why not just…?
Why not just make the injections realistic enough that no correction is needed? This is a real threat. If the richer injection model drives $\delta$ to zero on all three pairs, the correction is unnecessary and we will say so. The method still costs only a few real examples. Its value then becomes a check that certifies realism rather than a fix. Real tails, however, carry clumps, cirrus overlaps and host asymmetries that parametric injections plausibly miss. AutoProf found two-component parametric models frequently inadequate even for bright late-type galaxies. The injection-model ablation settles the question directly.
Why not just collect more confirmed real structures? Deep overlap fields are small, and the faintest confirmations need resolved star counts that work only nearby (Euclid Early Release Observations; A Tale of a Tail). Small real sets are therefore the operating condition, not a temporary inconvenience. The real-only baseline quantifies how wide the intervals become at $n$ = 10–40. The biomarker literature shows the parallel cost: Machine learning approaches to the human metabolome in sepsis found specificity ranging from 0.619 to 0.821 across models on 60 patients.
Why not just adapt the detector across surveys and trust its accuracy? Domain adaptation improves a detector but does not certify it. DeepAstroUDA's reported gains were measured with labels on bright morphologies, and its entropy threshold is a heuristic, not a calibrated probability. We include adaptation as a detector and test whether its entropy predicts misses. If adapted detectors come with calibrated self-assessment, our correction adds nothing for them, and the experiment will show it.
Why not just use the Gaussian-process posterior as the completeness? Star–Galaxy Image Separation showed that its posteriors flag ambiguous objects within one survey. However, it assumed balanced classes and did not test cross-survey shift. A posterior can be confident and wrong when the input distribution moves, so we measure its coverage against the held-out truth alongside ours.
Proposed Experiments
The main experiment repeats, 500 times per pair and detector, a random draw of $n \in \{10, 20, 40\}$ real calibration structures. For each draw we compute the three completeness estimates and record, in each 0.5 mag surface-brightness bin, three quantities: whether the 90% interval covers the held-out truth, the interval width, and the absolute error. The truth is exact in the LSST mocks and is the large held-out deep-confirmed set elsewhere.
The decisive outcome has two parts. First, the corrected estimator should hold coverage between 85% and 95% at $n = 20$ across all detectors. Second, over the same draws, injection-only should fall well below nominal coverage at the faint end, and real-only should be at least twice as wide.
The transfer experiment fits $\delta$ on SDSS/Stripe 82 and applies it unchanged to the LSST mock pair and to the degraded HSC pair. It scores coverage there, testing whether the realism gap belongs to the injection model or to the survey.
The proposal fails if coverage breaks even with the correction, because a two-parameter logit shift cannot capture how real and synthetic structures differ. The spline ablation shows whether that failure comes from the model's form or from the number of real examples, which tells the field which of the two it needs more of.
References
- A tale of a tail: a tidally disrupting ultra-diffuse galaxy in the M81 group
- APOC1 as a novel diagnostic biomarker for DN based on machine learning algorithms and experiment
- DeepAstroUDA: semi-supervised universal domain adaptation for cross-survey galaxy morphology classification and anomaly detection
- Detection, Instance Segmentation, and Classification for Astronomical Surveys with Deep Learning (DeepDISC): Detectron2 Implementation and Demonstration with Hyper Suprime-Cam Data
- AutoProf – I. An automated non-parametric light profile pipeline for modern galaxy surveys
- Star–Galaxy Image Separation with Computationally Efficient Gaussian Process Classification
- Euclid: Early Release Observations – Globular clusters in the Fornax galaxy cluster, from dwarf galaxies to the intracluster field
- Machine learning approaches to the human metabolome in sepsis identify metabolic links with survival