Genomes contain mobile DNA elements (transposable elements, or TEs) that copy themselves into new locations, and sometimes a new copy lands inside an older one. A region can then look like a sentence with clauses nested inside clauses: one element is interrupted by a second, which may itself be interrupted by a third. Current pangenome pipelines, which genotype variation against a graph of many human or plant genomes rather than a single reference sequence, report these nested insertions as a main failure mode. Neural sequence models trained on the single human reference are a separate weak point, because that reference comes about 70% from one donor and lacks much of the population's sequence. This proposal argues that two different failures are being blamed on one cause. The first is reference bias: the model has never seen the sequence. The second is depth failure: the model has seen every piece but cannot handle one more level of nesting than it was trained on. Work on formal grammars shows that the second failure does not go away with more context or more data at shallow depth.
We propose a factorial benchmark that crosses sequence novelty with nesting depth for TE insertions. We also propose a model that handles nesting by recursion in the procedure rather than in the network. A learned "peeler" finds and cuts out the innermost complete insertion, and the procedure repeats until nothing is left. The network only ever has to solve the depth-one problem, so depth generalization should follow from how the procedure is built, and its quality depends on how errors add up across steps. The claim to test is specific. Adding pangenome sequence to training should fix the novelty axis and leave the depth axis unchanged. Recursive peeling should fix the depth axis and leave the novelty axis unchanged. Only the two together should solve deep, divergent nests.
Why this intersection, and what already exists
Work on recursive generalization supplies the motivating evidence. COGS built a semantic-parsing benchmark from a probabilistic grammar. It found that sequence-to-sequence models trained to recursion depth 2 score near zero at depth 3 or more, even when input length is within the training range. SLOG isolated structural cases and reported 0% accuracy on center-embedding for a vanilla Transformer, T5-base and fine-tuned LLaMA-7B. In the same paper, a parser that predicts intermediate trees reached 70.8% overall. Unraveling Syntax trained small Transformers on nested-parentheses grammars and found length generalization intact while depth errors rose to 0.17. Together these establish that length and depth are separate axes of difficulty.
Two further papers constrain the design. Learning Syntax Without Planting Trees showed that hierarchical behavior in Transformers is seed-dependent and competes with a linear shortcut that emerges first in training. We therefore report all results across seeds and do not trust single runs. THEIA showed that a modular network applying discrete steps extrapolates from 5-step to 500-step chains at 99.96% ± 0.04%. That is the strongest evidence that recursion placed in the procedure, rather than in one flat pass, survives large depth extrapolation. We borrow this principle, not the architecture.
On the genomics side, GraffiTE genotypes TE insertions as bubbles in a variation graph. It reports more than 90% precision and recall on simulated human chromosome 22 and more than 94% recall on maize chromosome 10, and its authors name nested TEs and LTR–LTR recombination as the cases it misclassifies. HyenaDNA is a long-convolution DNA model with million-nucleotide context, pretrained only on hg38. It inherits the reference bias described by Ivanov et al. and Dewey et al., and it reads DNA as a flat string. VG-Pedigree pairs graph mapping with a convolutional caller but targets small variants and misses large structural ones. No paper in our file measures how any genomic model's accuracy changes with nesting depth, or separates depth from novelty. That measurement and the peeling model are what is new here. Building a "pangenome foundation model" is not the claim.
Core Claim
For nested TE annotation, a flat model's error as a function of depth beyond its training depth will be largely unchanged by adding non-reference sequence to pretraining. A peeler trained only on single insertions will hold accuracy across depths, with per-depth accuracy close to $(1-e)^d$. Here $e$ is the peeler's single-step error rate and $d$ is nesting depth, and the form assumes errors at each step are roughly independent. Where accuracy falls below that curve, errors are compounding: a wrong cut corrupts the outer element. That gap is the quantity we analyze.
Research Program
The benchmark has two axes. Depth runs from 0 to 4. Novelty has three levels. At the first level, the TE family and background are present in hg38. At the second, the background comes from non-reference sequence recovered in the 539-genome nonreference study or from the T2T 65-genome assemblies. At the third, the inserted TE copies are diverged by a controlled mutation rate, or come from a TE family held out of all training.
Simulated nests are built by inserting TE copies, with target-site duplications, into backgrounds drawn from the 94 HPRC haplotypes described in The Human Pangenome Project. Each further level is inserted at a random position inside an already-inserted element. The same protocol is applied to maize backgrounds, where TEs make up about 80% of the sequence. The label for each window is a tree of insertions, with family, span and orientation for each node. Real-data validation uses the 13 complex variants GraffiTE resolved at the maize bz locus. It also uses HPRC loci that GraffiTE flags as nested or misclassified, checked by hand against long-read assemblies.
The peeler is a HyenaDNA backbone with a span-and-family head. It is trained only on windows containing a single complete insertion, possibly inside a host that is itself an older TE. At inference it predicts the innermost complete insertion, cuts it out together with one copy of its target-site duplication, and repeats. It stops when it predicts "no complete insertion," which is an explicit abstain output so that uncertain calls are not forced. We train two backbones. One is the original hg38 checkpoint. The other is the same checkpoint with continued next-token pretraining on HPRC haplotypes for an equal number of tokens. This lets the novelty fix and the recursion fix be switched on and off independently.
Baselines
The naive reference point is a fine-tuned flat model. It is the hg38 HyenaDNA backbone with a single-pass tagging head that labels every nucleotide with its family and nesting level, trained on depths 0–2 and tested on 0–4. It is the obvious thing a practitioner would build, and COGS and SLOG predict it will hit a wall at depth 3.
The strongest existing system is GraffiTE, run with its published three-stage pipeline: assembly-based discovery, RepeatMasker annotation, and graph bubble genotyping. It is run on the identical simulated haplotypes, chromosomes 22 and 10, with default settings and the same compute allowance. Its reported figures are more than 90% precision and recall on human chromosome 22 and more than 94% recall on maize chromosome 10 (GraffiTE). Those figures are not broken down by depth, and there is no reported result found on Lacuna for depth-stratified nested-TE accuracy for any method. Beating GraffiTE means matching its depth-0 precision and recall within a point while exceeding its exact-tree accuracy at depths 2 and above, on human and maize separately.
A second strong competitor is classical iterative masking: run RepeatMasker, cut out the best-scoring hit that is complete and inside another hit, and repeat. This is a non-learned peeler. It is the real threat, because it is recursive by construction.
Two flat-model ablations isolate the key ingredients. The first is the flat model with the pangenome backbone, which tests whether novelty fixes depth. The second is the flat model trained on all depths 0–4, which serves as an oracle showing how much of the gap is pure extrapolation. The peeler with the hg38 backbone isolates recursion without the novelty fix.
All learned models get the same token budget, the same hyperparameter search of 20 configurations, and five seeds each. Results are reported as mean and spread, following the seed sensitivity documented in Learning Syntax Without Planting Trees.
Why not just…?
Why not just pretrain on the pangenome instead of the hg38 reference? This fixes exactly what reference bias is: sequence the model has never seen. If it also fixed nesting, our proposal would be unnecessary. The evidence from COGS and Unraveling Syntax is that depth failure persists even when every constituent appears in training, which predicts it will not. The flat-model-with-pangenome-backbone baseline settles this directly. If its depth curve beyond depth 2 matches the peeler's, the recursion half of our claim is wrong.
Why not just train the flat model on deeper simulated nests? Simulation can produce any depth, so extrapolation might seem beside the point. Two things keep it relevant. Real deep nests are rare, and the simulator's insertion process is itself an assumption, so a model that only interpolates is tied to the depths and patterns we chose to simulate. We include this oracle and report the result honestly. If the all-depth flat model matches the peeler at depth 4 and on the real bz and HPRC loci, the practical recommendation becomes "simulate deeper," and our contribution shrinks to the benchmark.
Why not just iterate RepeatMasker masking? This is the most serious alternative, because it already has the right recursive shape. Its weakness is plausible but untested in our file. When an inner insertion splits an older outer element, the two halves may each fall below homology thresholds, particularly for diverged copies. The learned peeler might be more robust because it sees the whole window. The third novelty level, with diverged and held-out families, is designed to expose this difference. If classical peeling matches the learned peeler there, we will report that the gain comes from recursion and not from learning.
Proposed Experiments
The first experiment is the factorial depth-by-novelty grid on human and maize simulations. The metrics are exact-tree accuracy, per-depth span F1, and outer-element reconstruction: after cutting out the inner insertion, does the outer element align to its family consensus? Two outcomes would be decisive. First, the flat models should show a depth wall that persists after pangenome pretraining, while peelers should not. Second, there should be an interaction in which only the pangenome-backbone peeler holds accuracy in the deep-and-novel corner.
The second experiment fits the $(1-e)^d$ curve, with $e$ measured at depth 1, and locates where observed accuracy falls below it. We check whether those failures concentrate at target-site-duplication boundaries or in LTR elements, the recombination case GraffiTE's authors flag.
The third experiment transfers to real loci, the maize bz region and the curated HPRC nested loci. The criterion is concrete: the peeler must recover trees that GraffiTE misclassifies, without increasing false insertions at depth 0.
A negative result on the first experiment would also be informative. If pangenome pretraining alone removes the depth wall, then on this problem "reference bias" and "depth failure" are one phenomenon, and the grammar literature's separation of length and depth does not carry over to DNA.
References
- COGS: A Compositional Generalization Challenge Based on Semantic Interpretation
- SLOG: A Structural Generalization Benchmark for Semantic Parsing
- Unraveling Syntax: Language Modeling and the Substructure of Grammars
- Learning Syntax Without Planting Trees: Understanding Hierarchical Generalization in Transformers
- THEIA: Learning Complete Kleene Three-Valued Logic in a Pure-Neural Modular Architecture
- A unified framework to analyze transposable element insertion polymorphisms using graph genomes
- HyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide Resolution
- AStarix: Fast and Optimal Sequence-to-Graph Alignment
- Phased Whole-Genome Genetic Risk in a Family Quartet Using a Major Allele Reference Sequence
- A complete pedigree-based graph workflow for rare candidate variant analysis
- Human pangenome analysis of sequences missing from the reference genome reveals their widespread evolutionary, phenotypic, and functional roles
- Complex genetic variation in nearly complete human genomes
- The Human Pangenome Project: a global resource to map genomic diversity