Pilot-Limited Polishing: How to Split a Sequencing Budget Between Long and Short Reads

Hybrid genome assembly rebuilds a genome from two kinds of sequencing data. Long reads (tens of thousands of letters, but with 1–15% errors) set out the large-scale layout. Short reads (a few hundred letters, more than 99.9% accurate) then "polish" that layout base by base. Labs choose how much of each to buy by rule of thumb. This proposal argues that the choice is the same resource problem wireless engineers solve when they split a channel's coherence block between pilots and data. Pilots are known training symbols used to learn the channel; the coherence block is the short window over which the channel stays fixed. If too few symbols go to pilots, the channel estimate is poor, and no amount of data transmission can recover. Our central claim is that short-read polishing behaves the same way. Short-read polishing quietly assumes the long-read layout is correct, the way many wireless designs assume perfect channel state information. When the layout collapses or misplaces repeat copies, short reads from different copies land on the same place and corrupt each other's consensus. This produces an error floor that extra short-read depth cannot remove.

If this floor exists and can be predicted from a genome's repeat structure, two things follow. The optimal long/short split can be computed before most of the budget is spent, from a small preliminary run. And the split is set by the genome's repeats rather than by habit. In two sentences: hybrid polishing is limited by the quality of the long-read "channel estimate," just as data rates are limited by pilot quality, so short-read error stops falling at a floor set by long-read coverage. We model that floor from repeat statistics and use it to choose the budget split, then test whether the rule matches a brute-force optimum on genomes it was not tuned on.

Core Claim

We model the per-base residual error of a polished hybrid assembly as the sum of two parts. The first is a statistical term that shrinks with short-read coverage. The second is a floor set by how many repeat copies the long reads fail to bridge:

$$\epsilon(C_L, C_S) \approx \rho(C_L)\,\epsilon_{\text{mix}} + \frac{\alpha}{C_S}.$$

Here $C_L$ and $C_S$ are long- and short-read coverage depths, meaning the average number of reads covering each base. The quantity $\rho(C_L)$ is the fraction of repeat-resident bases whose copies remain unbridged at long-read coverage $C_L$. We compute it from the genome's repeat-length spectrum and the read-length distribution using a coverage-based bridging probability. The constant $\epsilon_{\text{mix}}$ is the per-base error caused when short reads from paralogous copies (near-identical repeated segments) are pooled. The constant $\alpha$ captures ordinary sampling error. The $1/C_S$ form comes from consensus voting over independent reads.

Under a cost budget $W = c_L C_L + c_S C_S$, where $c_L$ and $c_S$ are per-unit-coverage costs, minimizing $\epsilon$ gives an interior optimum that depends on the slope of $\rho$. This mirrors how Björnson, Larsson and Debbah derive the pilot fraction from the structure of estimation error in Optimizing Multi-Cell Massive MIMO for Spectral Efficiency. They found that half the coherence block should go to pilots. We do not expect the number "half" to carry over. What we expect to carry over is the structure: the optimum is interior, and it is set by the "estimation" side of the budget.

The analogy is more specific than a slogan. Pooling short reads from unresolved repeat copies is the assembly version of pilot contamination, where reused pilot sequences corrupt channel estimates across cells Pilot Contamination Problem in Multi-Cell TDD Systems. The resulting floor is the analogue of the error floor in successive interference cancellation under imperfect channel knowledge described in Signal Processing for MIMO-NOMA. In that setting, adding transmit power cannot fix a bad subtraction. In ours, adding short reads cannot fix a bad layout.

Relation to Prior Work

The closest theory is Do Read Errors Matter for Genome Assembly?. It shows that noisy reads need only modestly longer lengths ($\tilde{\ell}_{crit} \approx \ell_{crit} + 3D$) to reconstruct a genome. Its setting is the dense-read limit, with one homogeneous read type and infinite coverage. This result motivates our model: it says the repeat spectrum, not base accuracy, governs resolvability, which is why $\rho$ is written in terms of repeats. It also constrains our model: we reuse its per-genome critical lengths as a sanity check on $\rho$. We depart from it by working at finite coverage, with two read types and an explicit cost.

HINGE separates bridged repeats from unbridged ones directly on noisy reads. We reuse its idea of an explicit unbridged-repeat set as the empirical measurement of $\rho(C_L)$. HINGE itself says nothing about how many short reads to buy.

The neural assemblers show that graph untangling, not base accuracy, is the dominant failure. Learning to Untangle Genome Assembly with Graph Convolutional Networks classifies which edges of an assembly graph lie on the true genome path. A step towards neural genome assembly uses a message-passing network to reproduce standard graph-simplification heuristics. Both take the reads as fixed input. We treat the read mix as the decision variable and hold the assembler fixed.

On the wireless side, Vision Transformer for Adaptive Image Transmission over MIMO Channels and Estimation Diversity and Energy Efficiency in Distributed Sensing are cautionary examples of strong designs derived under perfect channel knowledge. We use them only as motivation.

Research Program

The work has three parts.

The first part measures the floor. We simulate hybrid read sets from finished references over a grid of $(C_L, C_S)$ and long-read error rates, and assemble each with a fixed hybrid pipeline. The references are the GAGE bacteria, including S. aureus with its reported $\ell_{crit}$ of 1,799 bp, and CHM13 chromosomes held out from any tuning. We record consensus error separately for unique regions and repeat-resident regions. The central prediction is that in repeat regions error plateaus in $C_S$ at a level that falls with $C_L$, while unique regions follow $1/C_S$.

The second part fits $\rho$ and $\epsilon_{\text{mix}}$ from quantities available before full sequencing. These are the repeat spectrum of a quick draft assembly and the long-read length distribution. From them we derive the cost-optimal split.

The third part tests the rule prospectively, the way a pilot-length rule is tested at a new coherence time. For each held-out genome we take a small preliminary sample (a few percent of the budget), predict the split, spend the rest of the budget accordingly, and compare against the true optimum found by exhaustive downsampling.

Baselines

The naive reference point is the fixed-ratio practice used in hybrid case studies such as Genomic characterization of an emerging Enterobacteriaceae species. Those studies choose coverages ad hoc. We represent this practice as a small set of constant long:short cost ratios, run on exactly the same simulated budgets.

The long-only reference point is a single-technology assembly spending the whole budget on long reads. The strongest long-only method in the file is HINGE, which reports 782 finished assemblies on the NCTC 3000 collection versus 592 for the standard NCTC manual pipeline HINGE. We run it on the same budgets and count finished, reference-correct assemblies. Where layout quality matters on human data, we note that the graph-network untangler reports 26 contigs versus 241 for Raven on CHM13 chromosome 1 Learning to Untangle. This is a layout benchmark, not an allocation one.

The strongest competitor on the actual question is an oracle grid search: exhaustive downsampling over $(C_L, C_S)$ on the target genome itself, with unlimited compute, reported as the best achievable error at each budget. No reported result was found on Lacuna for any hybrid long/short allocation benchmark, so this oracle is computed by us. Beating it is impossible by construction. Our rule counts as successful if its chosen split lands within a small, pre-registered excess error of the oracle on held-out genomes, at a small fraction of the oracle's sequencing and compute cost.

The key ablation is a floor-free model that sets $\rho \equiv 0$, which amounts to assuming a perfect backbone. This model predicts that short reads always help. It isolates whether the pilot-contamination term is what makes the rule work.

Why not just…?

Why not just grid-search the coverage mix by downsampling? On a genome that has already been sequenced deeply, a grid search wins trivially, which is why it is our oracle. The practical question is choosing the split before the money is spent, on a genome with an unknown repeat structure. A grid search tuned on one genome does not transfer to another, because the optimum should track the repeat spectrum. We test this directly. We apply the best split from one genome to others and compare the transfer error against our rule's prospective predictions. If transferred grid optima do as well as our rule, the modeling adds nothing and we will say so.

Why not just use PacBio HiFi reads and skip hybrid assembly? High-accuracy long reads remove most base errors, but not repeat ambiguity. Raven on real HiFi data still produced 241 contigs on CHM13 chromosome 1 Learning to Untangle. Noisy ultra-long reads remain the tool used in large field surveys such as Cartography of opportunistic pathogens. The floor model applies unchanged to HiFi if we set its per-base error low. We include a HiFi-only arm at matched cost to check whether hybrid allocation still matters there. A null result would narrow our claim to noisy long reads.

Why not just add more short-read depth or use a better polisher? This is the claim most directly at risk, and the floor experiment settles it. If repeat-region error keeps falling as $1/C_S$ at every long-read coverage, there is no floor and the pilot analogy fails. We also run a repeat-aware polisher that ignores reads with multiple placements. This is the analogue of contamination-avoiding pilot scheduling Pilot contamination reduction in massive MIMO systems based on pilot scheduling. The prediction is that it lowers $\epsilon_{\text{mix}}$ but leaves bases in unbridged copies uncorrected, so the floor persists at a lower level.

Why not just use the dense-read bound directly? $\tilde{\ell}_{crit}$ tells us whether a read length can ever suffice. It says nothing about finite-coverage trade-offs between two read types Do Read Errors Matter. We use it as a boundary condition: when read lengths exceed $\tilde{\ell}_{crit}$, $\rho$ should vanish at high $C_L$.

Decisive Checks

The first check is a pre-registered plateau test on the GAGE and CHM13 simulations. For repeat-resident bases, the fitted exponent of error against $C_S$ must fall toward zero above a coverage threshold, and that threshold must shift with $C_L$. The second check is predictive accuracy of the floor-aware rule versus the floor-free ablation, measured as excess error over the oracle on held-out genomes. Assembly correctness throughout is scored by finished-genome count on NCTC-style bacterial data and by misassemblies and consensus errors against references. If the floor is absent or the ablation predicts as well, the pilot-contamination account is wrong and the allocation rule reduces to ordinary coverage arithmetic.