Forgetting Easy Decoys: Competence-Aware Memory Pruning for Genomic Site Detection

Most of the DNA that looks like a functional site is not one. In the human genome, the two-letter signals that mark where a gene's coding pieces are cut and rejoined (splice sites) occur about a thousand times more often by chance than as real cut points. Any system that predicts by looking up similar, already-labeled examples is flooded with these lookalike "decoys." This proposal attaches a lookup memory of labeled DNA windows to a frozen DNA foundation model, meaning a large network pre-trained on raw genomes whose weights we do not update. It then deliberately forgets entries the model already handles. The forgetting rule is borrowed from machine translation. An entry is deleted when a cheap predictor built on the frozen model is correct both on that entry and on its whole neighborhood in embedding space. Entries near the boundary between real sites and decoys are kept. Because real sites are so rare, the rule is made class-asymmetric, with a stricter bar for forgetting a rare true site than a common decoy.

The point is not that a lookup memory is a new genomic classifier. The point is to answer a concrete question no one has tested: under 1,000:1 decoy imbalance, which examples should a bounded memory forget? The test is whether a rule based on neighborhood competence beats the two obvious answers, random downsampling and standard hard-negative mining, at equal memory size. The memory is also editable, so a third question is whether it can absorb data from a new species and shed redundant data without any retraining.

Core Claim

Our claim is that a neighborhood-level competence rule produces a much smaller genomic datastore with no loss in precision–recall on splice-site detection, and that it does so where per-example difficulty does not. The rule is class-conditional, applied over frozen DNA-model embeddings. By "per-example difficulty" we mean keeping the examples the predictor gets wrong or is unsure about, which is what hard-negative mining does. If the competence rule cannot beat per-example hard-negative selection at matched size, the claim fails.

The method comes from What Knowledge Is Needed? Towards Explainable Memory for kNN-MT Domain Adaptation. That paper defines a "knowledge margin": how far around a datastore entry the base translation model stays consistently correct. Its pruning rule, PLAC, removes high-margin entries and cuts translation datastores by up to 45% with no BLEU or COMET loss. It also reports that random pruning hurts quality. PLAC was built for token prediction over a text vocabulary with roughly balanced targets. It says nothing about rare-positive classification, where removing high-margin entries of the majority class could empty the memory of the very decoys that teach the boundary.

The genomic motivation comes from Accurate splice site prediction using support vector machines, which documents the 1,000:1 decoy ratio and shows that position-aware sequence kernels are needed to separate real sites from decoys. Benchmarking DNA foundation models for genomic and genetic tasks supplies the representation choice. Averaging all token embeddings ("mean pooling") beats using a single summary token by 1.4% to 8.7% AUC, and Caduceus-Ph is the strongest model on human genome classification. That benchmark also cautions that fine-tuning choices can obscure what frozen models actually encode. This is one reason we keep the encoder frozen.

Research Program

The system has three parts.

Frozen encoder. Caduceus-Ph, NT-v2 and HyenaDNA each embed short windows centered on candidate splice dinucleotides. Embeddings use mean pooling.

Base predictor. A class-weighted logistic probe is trained once on these embeddings. It plays the role of PLAC's base model, which is needed because no DNA foundation model has a native splice head.

Datastore and prediction. The datastore holds embedding–label pairs. A query's score interpolates the probe's probability with a distance-weighted vote over its retrieved nearest neighbors, as in nearest-neighbor translation. The interpolation weight and the number of neighbors are tuned on validation data.

For each entry $i$ with label $y_i$, we define its margin $m_i$ as the largest $k$ such that the probe correctly classifies $i$ and all of its $k$ nearest neighbors in the store. This is a neighbor-count version of PLAC's radius. Entry $i$ is forgotten when

$$ m_i \ge \tau_{y_i}, \qquad \tau_{+} > \tau_{-}, $$

where $\tau_{+}$ is the threshold for true sites and $\tau_{-}$ is the threshold for decoys. Both thresholds are chosen on validation data to meet a target store size.

The asymmetry is the genomic adaptation. Decoys sitting deep inside regions the probe already rejects carry no information. True sites are scarce enough that only the most redundant ones should go. Margins are recomputed after each pruning round, because deleting neighbors changes the margins of the entries that remain.

The analysis traces what survives pruning: decoys that share sequence near the junction, positives with weak signals, and how surviving entries spread across positions relative to the junction. The second phase is a streaming test of editability. The store is built on C. elegans splice data, and human data is then appended. Margins are recomputed and pruning is re-run with the probe frozen. We then measure whether cross-species redundancy is shed without hurting either species.

Baselines

Probe alone (do-nothing reference). The naive reference is the class-weighted probe with no datastore. Next to it sits the unpruned full datastore, which shows what retrieval adds before any forgetting. Both use the same frozen embeddings, splits and validation-based tuning. If the full store does not beat the probe, retrieval is not worth studying here.

Matched-size pruning baselines. Four baselines are compared at the same number of retained entries, swept from 100% down to 10% of the training set:

  • random downsampling;
  • class-stratified downsampling that keeps all positives and randomly thins decoys;
  • PLAC applied as published, with a single class-agnostic threshold;
  • per-example hard-negative selection, which keeps the entries with the highest probe loss.

The last one is the ablation that isolates our key ingredient: it uses the same probe and the same budget, but drops the neighborhood requirement. Beating it means higher auPRC at equal store size, with the gap holding across three seeds and all three encoders.

Classical sequence kernel. The strongest existing splice method in the file is the Weighted Degree kernel with shifts (a support vector machine that compares short subsequences position by position, with tolerance for small shifts), from Accurate splice site prediction using support vector machines. It reports an auPRC of about 54–57% on human splice sites, against about 16–25% for Markov-chain models. We will retrain it with the published toolbox on our exact splits, giving it a hyperparameter search budget comparable to ours. Matching it with a pruned store at a fraction of the training set counts as success. Exceeding it would be a bonus; we do not assume we will.

Secondary reference points. On the human RefSeq acceptor set, Features generated for computational splice-site prediction correspond to functional elements reports a false-positive rate of 2.49% at 95% sensitivity, against 5.80% for GeneSplicer. On the small C. elegans set, Support Vector Machines and Kernels for Computational Biology reports 98.9% auROC for the Weighted Degree kernel. For frozen DNA foundation model embeddings on splice-site auPRC, there is no reported result found on Lacuna, so those numbers are produced here.

Why not just…?

Why not just fine-tune the encoder or train a stronger probe, and skip memory entirely? This is the most serious threat. A well-weighted classifier on frozen embeddings may match any retrieval system on a fixed benchmark. We therefore make no claim that the datastore wins statically. We report the probe, a two-layer probe, and a parameter-efficient fine-tune at matched compute. If they match the pruned store, the remaining argument for memory is editability, which the streaming experiment tests directly: adding a species and forgetting redundancy with no gradient updates. The benchmark paper's warning about fine-tuning bias (Benchmarking DNA foundation models for genomic and genetic tasks) is a reason to keep a frozen-encoder comparison. It is not evidence that fine-tuning is worse.

Why not just do hard-negative mining? Keeping high-loss decoys is the standard answer to class imbalance, and it overlaps with our rule. The difference is that per-example loss deletes a confidently-rejected decoy even when its neighbors are misclassified. The neighborhood rule keeps such a decoy, because it anchors a region the probe gets wrong. The per-example ablation in the baselines tests exactly this. If the two rules perform the same, the honest conclusion is that PLAC's neighborhood margin adds nothing in genomics, and we would report that.

Why not just downsample decoys at random or stratify by class? In translation, random pruning measurably degraded quality where competence pruning did not (What Knowledge Is Needed? Towards Explainable Memory for kNN-MT Domain Adaptation). Genomics may differ, because embedding neighborhoods of decoys could be uniformly uninformative. The size-sweep curves settle this. Stratified downsampling is the version a practitioner would actually use, so it is the bar the competence rule must clear at small store sizes.

Why not just filter by evolutionary conservation? Conservation scores such as the Branch Length Score miss 14–19% of real motifs (Reliable prediction of regulator targets using 12 Drosophila genomes). They also answer a different question: whether a site is under selection, not whether the model needs it in memory. Conservation is complementary rather than competing, and we leave it as an optional added feature, not a baseline.

Decisive Experiments

Splice-site detection versus store size. The main experiment uses the human splice-site benchmark with genome-wide decoy ratios from Accurate splice site prediction using support vector machines. It plots auPRC against retained store size for every pruning rule and every encoder. The claim holds if class-conditional competence pruning keeps auPRC within one standard error of the full store at 50% size or less, and beats per-example hard-negative selection at every size below that. A second check verifies that the false-positive rate at 95% sensitivity on the RefSeq acceptor set does not degrade under pruning.

Calibrating the margin. The third check tests whether the margin means in DNA embeddings what it meant in translation. PLAC found that margins above 32 meant better than 95% held-out accuracy, and margins below 4 meant under 50%. We test whether held-out probe accuracy rises steadily with margin in our setting, separately for true sites and decoys. A flat curve would mean frozen DNA representations lack the neighborhood structure the method depends on. That would be a clean negative result in its own right.

Streaming across species. The C. elegans-then-human experiment reports per-species auPRC, store size and turnover after each append-and-prune round. The comparison is against retraining the probe from scratch on the combined data. Matching retraining with no gradient updates would establish the one property a static classifier cannot offer.