Selection-Aware Outlier Synthesis for Prototype Classifiers of Survey Transients

Telescope surveys train their classifiers on the objects they could afford to follow up spectroscopically. Those objects are bright and nearby, while the survey itself is dominated by faint, distant, noisy ones. Two jobs collide here. The classifier must accept a faint distant supernova as a supernova, and it must also flag a genuinely new kind of transient as unknown. Current outlier-detection methods train the second skill by inventing fake "outliers" just past the edge of the training data. Under survey selection bias, that edge is not a property of the physical class. It is a property of the follow-up budget. Fake outliers placed there land on the legitimate faint objects the survey most needs to keep.

We propose to make the edge condition-dependent. Each class is represented not by one prototype (a single reference point in the network's embedding space, as in nearest-centroid classifiers) but by a prototype trajectory. The trajectory traces how that class's embedding moves as the class is physically redshifted and degraded by the survey's own noise and cadence. Synthetic outliers are then generated only in directions that no change in observing conditions can explain. Novelty is scored as the distance that remains after the best-fitting observing condition has been accounted for. The claim, stated in two sentences: an anomaly is something that cannot be explained as a known class seen through the survey's observation process. Building that definition directly into prototypes and outlier synthesis should improve unknown-transient rejection without discarding faint, distant members of known classes.

Core Claim and Mechanism

Let $f_\theta$ be a light-curve encoder and $s$ an observing condition, for example redshift together with limiting depth. For class $k$ the prototype trajectory $c_k(s)$ is built in three steps. First, fit Gaussian-process curves to training light curves, the same fitting STACCATO uses. Second, transport each fit to condition $s$ by redshift-dimming, time dilation, survey-specific cadence and noise injection, and re-embed it. Third, average the embeddings within condition bins and smooth across bins with a small regression.

For each class and each condition stratum, a held-out calibration set gives the 95th and 99th percentiles $q_{95}(s), q_{99}(s)$ of the distance to $c_k(s)$. Synthetic outliers are drawn as

$$\tilde z = c_k(s) + r\,u,\qquad u \perp \mathrm{span}\{\partial c_k/\partial s,\ \text{top principal directions of class } k \text{ at } s\},\qquad r \in [q_{95}(s), q_{99}(s)].$$

In words, the perturbation leaves both the data manifold and the nuisance curve, and its size matches the class spread at that condition.

The novelty score is

$$A(x)=\min_k \min_s \frac{\lVert f_\theta(x)-c_k(s)\rVert}{q_{95}(s)}.$$

This score separates two cases that a fixed-point prototype cannot tell apart. A degraded member of a known class reaches a small $A$ by sliding along its trajectory. A true novelty stays far from every trajectory.

Training uses three terms. A distance-softmax cross-entropy pulls the transported examples to their condition-matched prototypes. A contrastive term pushes synthetic outliers beyond every trajectory. An optional $(K{+}1)$-th sink class absorbs those outliers.

Relation to Prior Work

The method draws on two sources. The outlier-placement recipe comes from Geometrically Constrained Outlier Synthesis (GCOS). GCOS perturbs penultimate-layer features along low-variance principal directions, keeps the perturbation size inside a calibrated "shell" of nonconformity quantiles, and trains an energy-regularized softmax head. We keep the shell idea but make three changes. The shell is indexed by observing condition, the nuisance directions are explicitly removed from the perturbation space, and the classifier is metric rather than linear.

The use of physical transport to fill sparse regions comes from STACCATO. STACCATO draws Gaussian-process light curves within propensity strata to augment under-represented in-distribution regions. It does no outlier rejection, and its classifier is a random forest. We use its transport step to define where known classes can legitimately appear, so that outliers can be placed everywhere else.

Prototype-plus-outlier coupling already exists. Training Auxiliary Prototypical Classifiers for Explainable Anomaly Detection in Medical Image Segmentation pushes handcrafted, extreme-transformation outliers away from prototypes on clean medical images. A Closer Look at Few-Shot Out-of-Distribution Intent Detection maps mixup interpolations between classes to an extra outlier class. Neither has a notion of a selection-truncated training set. Mixup also cannot extrapolate beyond the outer perimeter of the data, which is exactly where survey novelties sit.

Dream the Impossible argues for decoding boundary embeddings into realistic inputs. Its roughly 10-hour cost per 100k images makes iterative retraining expensive. GCOS's own comparison also favors feature-space synthesis on near-OOD tasks, reporting 93.47% average AUROC against 85.76% for Dream-OOD. We therefore stay in feature space and let the physics enter through transport.

The backbone choice follows Meta-Baseline, which pre-trains a strong encoder and then classifies by nearest centroid. Prototype classification itself is due to Prototypical Networks for Few-shot Learning. That backbone also lets newly confirmed classes be added at test time from a handful of examples.

The motivation comes from astronomy. Results of the PLAsTiCC challenge report that most top models misassigned the novel "Other" class to common classes such as Type Ia. More broadly, Dynamic or Systematic? shows that an unmodeled 0.04 mag calibration offset can masquerade as dynamical dark energy, with a log-Bayes factor of about 4.1 favoring the systematic explanation. Observational nuisance repeatedly gets mistaken for new physics, and an anomaly detector that cannot tell the two apart will repeat that mistake.

Research Program

The work has four parts. The first is the transport-and-embed pipeline, which turns each spectroscopically confirmed training light curve into a family of light curves across redshift and depth bins under the survey's noise and cadence model. The second is the trajectory-prototype head and the condition-stratified conformal calibration. The third is the projected outlier sampler.

The fourth part is an analysis of the failure the method targets. For plain GCOS trained on the biased training set, we will measure what fraction of its synthetic outliers fall within the empirical distance quantiles of true faint test members of known classes. This "outlier collision rate" is the direct measure of the problem we target. The prediction is that it is large for plain GCOS and near zero for our sampler.

We will also test a weaker assumption: that learned nuisance directions approximate the physical ones. To do this we compare tangents of $c_k(s)$ against embedding shifts produced by held-out degradations that were never used in transport.

Baselines

The naive reference is a prototype classifier trained on the biased set with a simple distance threshold for rejection, the open-set use of fixed prototypes described in the file. Comparing against it isolates whether outlier synthesis helps at all.

The strongest classification competitor in astronomy is a PLAsTiCC-style gradient-boosted ensemble on Gaussian-process features with redshift-dimming and noise augmentation. Its rejection score will be maximum class probability. It is the strongest because it set the challenge's best weighted log-loss, 0.685 (PLAsTiCC). For SPCC, the strongest reported classifier is STACCATO, with AUC 0.96 versus 0.93 for the biased set and 0.977 for the unbiased gold standard (STACCATO). There is no reported "Other"-class number for any PLAsTiCC entry on Lacuna, so all rejection numbers will be measured by us under a shared protocol.

The strongest outlier-synthesis competitor is GCOS, run twice: once on the biased features, and once on features after STACCATO-style transport augmentation. The second run, "augment-then-GCOS", is the critical baseline. Its reported vision result, 93.47% average near-OOD AUROC, does not transfer to light curves, so we rerun it.

Fairness rules apply to every method. All methods share the encoder architecture, the Gaussian-process fits, the transport simulator, the training data, and a fixed hyperparameter search budget tuned on the same validation split.

Three ablations isolate the key ingredients. The first removes the nuisance projection but keeps trajectory prototypes. The second keeps a single global shell instead of condition-stratified shells. The third replaces trajectories with one prototype per class trained on transported data. Beating a baseline means two things together. First, higher unknown-versus-known AUROC and lower FPR95. Second, no loss of the classification metric (log-loss on PLAsTiCC, AUC on SPCC) and no increase in the rejection rate of known-class objects in the highest-redshift and lowest-propensity bins.

Why not just…?

Why not just augment the training set with transported light curves and then run GCOS unchanged? This is the real threat. Augmentation inflates variance along nuisance directions, so PCA would partly exclude them from perturbation on its own. Our expectation is that it falls short in one specific way. A single class-level shell is too wide at high signal-to-noise and too narrow at low signal-to-noise, because class spread is heteroscedastic across conditions. That would make outliers either trivial near bright objects or colliding near faint ones. The "global shell" and "no projection" ablations, together with the collision-rate analysis stratified by redshift, settle this. If augment-then-GCOS matches us in every bin, the honest conclusion is that transport, not our geometry, does the work, and we will report it that way.

Why not just threshold distance to ordinary prototypes? Distance thresholds reject faint known objects and novelties alike, because both drift away from a centroid fit to bright data. Thresholds also leave the space between prototypes open, the overconfidence documented by Deep neural networks are easily fooled. The naive baseline measures exactly this cost, and the condition-indexed score $A(x)$ is designed to remove it.

Why not just use real auxiliary outliers, as in Deep Anomaly Detection with Outlier Exposure? No large auxiliary transient set exists that shares the survey's filters, cadence and noise. External data also tends to sit far from the boundary, where shortcuts suffice. A partial test is still possible. We will use held-out PLAsTiCC classes as "real" exposure in a variant and compare it against synthetic outliers on the remaining unknowns.

Why not just keep a tree ensemble with propensity weighting? It is a strong closed-world classifier, which is why it is a baseline. However, PLAsTiCC's qualitative finding is that such ensembles collapse unknowns into common classes, and they cannot absorb a newly confirmed class from a few examples without retraining.

Proposed Experiments and Decisive Checks

On SPCC we train on the 1,217-object biased set and evaluate on the 16,113-object test set. Unknowns come from leave-one-class-out: each non-Ia type is held out in turn. We report classification AUC against the 0.93/0.96/0.977 reference points, unknown-detection AUROC and FPR95, and known-class false-rejection rate per redshift and propensity bin.

On PLAsTiCC we train on the roughly 8,000-object training set and evaluate on the test set. The challenge's own novel classes serve as unknowns, and we additionally hold out several training classes. We report weighted log-loss alongside the same rejection metrics.

A third test uses controlled corruption. We inject survey-like degradations into known-class test light curves, either harsher than those used in transport or drawn from a degradation family outside the transport model. A method that merely learned "faint means suspicious" will flag these as anomalies, while ours should not. This directly probes the gap between systematic artifacts and physical anomalies.

Two outcomes would refute the proposal. The first is a high-redshift false-rejection rate for our method that is no better than plain GCOS. The second is unknown-detection performance that augment-then-GCOS fully matches across all bins. A third outcome would refute the mechanism specifically: nuisance tangents that fail to align with held-out physical degradations.