Differentiable Capture Probe Design for the Undiagnosed Genome

More than half of patients with a clear clinical presentation of a rare genetic disease leave the clinic without a molecular diagnosis after standard testing — in one 47-family cohort with cone-rod dystrophy, 78.72% of families remained undiagnosed despite exome sequencing of all 25 known causative genes (Exome Sequencing of 47 Chinese Families with Cone-Rod Dystrophy). A large part of the problem is physical, not informational: clinical sequencing relies on hybridization capture, in which short synthesized DNA strands called probes fish the regions of interest out of a genome sample. Because probe libraries are designed heuristically and post-filtered — probes with extreme GC content, stable hairpin folds, or placement in repetitive sequence are simply discarded — the resulting panels are structurally blind to the deep intronic, regulatory, and repeat-adjacent variants that repeatedly turn out to be the missed diagnosis, such as the deep intronic JMJD6 variant causing congenital eyelid coloboma (Case report: A novel intronic JMJD6 likely pathogenic variant (c.941+75G > T) associated with congenital eyelid coloboma in one of the identical twin sisters) and the non-coding blind spots catalogued in Beyond the exome: What's next in diagnostic testing for Mendelian conditions.

Engineering has already solved an isomorphic problem. In lithography, structural design, and nanophotonics, unconstrained optimizers produce designs that fail manufacturing, and post-hoc correction destroys the very physics the design was optimizing for — rounding an unconstrained photonic device to satisfy a minimum feature size collapses its resonance (Machine learning to optimize additive manufacturing for visible photonics). The fix was to move the constraints inside the optimization: Differentiable Edge-based OPC embeds lithographic forbidden zones directly into gradient velocities, Diffusion Models Beat GANs on Topology Optimization uses surrogate classifiers to kill disconnected structures during generation, cutting the floating-material failure rate from roughly 68% to 6%, and Shaping freeform nanophotonic devices with geometric neural parameterization enforces feature size and connectivity as differentiable penalties. This proposal applies that same proactive-to-reactive shift to the design of clinical capture panels: treat the probe library as a constrained inverse design object, with biophysical synthesis rules embedded as differentiable penalties, so that clinically critical "dark" regions are captured by construction rather than rescued by filtering.

Core Claim

Capture panel design can be formulated as a continuous, differentiable optimization in which probes are movable genomic intervals and biophysical synthesis rules (uniform melting temperature $T_m$, GC-content windows, terminal-GC stability, hairpin and cross-hybridization avoidance, repeat placement) act as differentiable penalty fields and forbidden-zone gradient scaling — the direct genomic analogue of the machinery in Differentiable Edge-based OPC and Shaping freeform nanophotonic devices with geometric neural parameterization. The claim is that this design regime yields custom panels covering non-coding and repeat-rich disease loci at uniform depth with near-zero constraint violations, where post-filtered heuristic designs — the current clinical standard — leave coverage dropouts exactly where the missed diagnoses live.

Two properties distinguish this from all prior work on genomic probe design. First, Design optimization methods for genomic DNA tiling arrays solved repeat avoidance for legacy microarrays with a linear-time dynamic program over binary repeat masks and a heuristic "bridge strategy"; it never modeled hybridization thermodynamics, never handled modern sequencing capture, and was never evaluated on diagnostic yield. Second, Stable DNA Sequence Over Close-Ending and Pairing Sequences Constraint enforces thermodynamic constraints on de novo DNA-computing sequences with a discrete whale metaheuristic that cannot scale to genome-wide probe libraries. Neither work — nor the purely descriptive platform comparison of Performance comparison of four exome capture platforms on DNBSEQ-Series high throughput sequencer — makes the design loop differentiable or connects constraints to clinical blind spots.

Research Program

The first build is the parameterization. A panel is a set of probes, each a genomic interval of fixed synthesis length (e.g., 120 bases) with a center coordinate $c_i$ and a sequence assignment. We optimize a continuous coverage field over a clinically prioritized target locus (for example, the intronic and regulatory neighborhoods of a known disease gene), where each base $b$ carries a weight $w(b)$ reflecting diagnostic relevance — known pathogenic non-coding sites, predicted splice-disrupting positions, and structural-variant breakpoint zones score high, following the blind-spot taxonomy of Beyond the exome: What's next in diagnostic testing for Mendelian conditions. The optimization problem is:

$$\min_{\theta}\; \underbrace{\sum_{b} w(b)\,\big(1 - C_\theta(b)\big)}_{\text{weighted uncovered bases}} \;+\; \lambda_m\,\mathrm{Var}_i\!\big[T_m(p_i)\big] \;+\; \lambda_h \sum_i \max\!\big(0,\; \Delta G^{\text{hairpin}}_i - \Delta G^{*}\big) \;+\; \lambda_x\, \mathcal{C}_{\text{cross}}(\theta),$$

where $\theta$ collects the probe centers, $C_\theta(b)$ is a differentiable coverage indicator (a soft window around each center), the second term penalizes melting-temperature variance across the library, the third penalizes any probe whose predicted hairpin free energy $\Delta G^{\text{hairpin}}_i$ is more stable than a synthesis threshold $\Delta G^{*}$, and $\mathcal{C}_{\text{cross}}$ is a soft kernel counting near-complementarity between probes and off-target sites — a differentiable rendering of the pairing-sequence constraint from Stable DNA Sequence Over Close-Ending and Pairing Sequences Constraint. Hairpin and cross-hybridization energies come from differentiable relaxations of the nearest-neighbor thermodynamic models behind NUPACK, which that paper used only as a non-differentiable post-hoc evaluator.

The second build is forbidden-zone velocity scaling, lifted from Differentiable Edge-based OPC. Rather than adding ever-larger penalties, each gradient step that moves a probe center is multiplied by $\exp(-\rho(c_i)/\tau)$, where $\rho(c)$ is a continuous "violability density" at genomic coordinate $c$ — built from GC extremity, local repeat density (a smooth rendering of RepeatMasker annotations rather than a binary mask), and predicted off-target complementarity. Gradients decay to zero as a probe approaches a region where no compliant sequence exists, so the trajectory curves away before a violation, and nucleotide hard assignments use a straight-through estimator for the rounding step, exactly as DiffOPC does for edge rasterization. Crucially, because $\rho$ is continuous rather than binary, the optimizer can find the minimal controlled intrusion into difficult regions — the thermodynamic generalization of the bridge strategy in Design optimization methods for genomic DNA tiling arrays, which showed that strictly skipping every repeat costs roughly 19–20% coverage while a controlled bridge keeps repeat inclusion near 4%.

The third build is surrogate guidance in the style of Diffusion Models Beat GANs on Topology Optimization, which used a compliance regressor and a floating-material classifier to steer generation. We train two surrogates on public capture-sequencing data: a per-probe capture-efficiency regressor (predicting on-target depth from sequence features and genomic context) and a cross-hybridization classifier (predicting off-target dominance). During optimization, the regressor's gradients improve expected uniformity — addressing the GC-driven coverage dropouts that Performance comparison of four exome capture platforms on DNBSEQ-Series high throughput sequencer showed differentiate a good panel (Twist Exome 2.0: 98.00% coverage uniformity at 30x, fold-80 penalty 1.31) from a poor one — while the classifier's gradients suppress probes that would steal reads from off-target loci.

The final build is the clinical target selection layer. We will design custom panels for three documented blind-spot classes: the deep intronic splice-disrupting region around JMJD6 c.941+75G>T (Case report: A novel intronic JMJD6 likely pathogenic variant (c.941+75G > T) associated with congenital eyelid coloboma in one of the identical twin sisters); repeat-flanked structural-variant breakpoint neighborhoods of the type that short-read panels miss (EXT1 and EXT2 Variants in 22 Chinese Families With Multiple Osteochondromas: Seven New Variants and Potentiation of Preimplantation Genetic Testing and Prenatal Diagnosis, where 18% of families remained unexplained); and the full non-coding extent of the 25 cone-rod dystrophy genes whose coding-only capture left 78.72% of families undiagnosed (Exome Sequencing of 47 Chinese Families with Cone-Rod Dystrophy). Downstream variant ranking is deliberately out of scope: Refined preferences of prioritizers improve intelligent diagnosis for Mendelian diseases achieves top-10 ranking of the causal gene in 83.3–91.8% of trios, but only for variants that were actually sequenced. Our claim is strictly about making the causative variant readable.

Proposed Experiments

The evaluation runs in three stages. Stage one is in-silico: for each target locus, we synthesize probe libraries with our method and all baselines, then run a thermodynamic capture simulator (nearest-neighbor hybridization over the target locus plus a decoy background) to predict per-base depth. Metrics are the standard panel-quality measures used by Performance comparison of four exome capture platforms on DNBSEQ-Series high throughput sequencer: coverage uniformity at 30x depth, fold-80 base penalty, and off-target rate, plus repeat-region base coverage and the fraction of constraint-violating probes at the end of design (Tm outliers, stable hairpins, terminal AT instability). No reported result found on Lacuna for off-target rates of custom non-coding panels, so we establish our own reference distribution rather than compare to an invented figure.

Stage two is wet-lab: the two best designs per locus are synthesized and run on the NA12878 reference standard, the same benchmark and metrics (Q30 rates, ~99.7% expected mapping rate, SNP precision exceeding 99% for all platforms) used by Performance comparison of four exome capture platforms on DNBSEQ-Series high throughput sequencer, and on the PancancerLight 800 standard, whose 793 expected mutation sites across 1% to 100% allele frequencies provide the low-allele-frequency sensitivity check. For the NA12878 non-coding targets, the comparison is against our own baselines (below), since commercial exomes contain no probes there by construction — that absence is precisely the diagnostic gap.

Stage three is the decisive clinical check: for the CORD locus panel, we ask whether the previously unexplained families' candidate regions — including the intronic and regulatory space of the 25 genes — achieve capture depth sufficient to call variants in-silico, using the coverage simulator calibrated in stage two. This is a coverage-yield analysis, not a claim of new diagnoses, and it directly tests whether the 78.72% undiagnosed fraction could be attacked with better probes rather than new genes.

Baselines

The "do nothing" reference point is the current clinical standard: a commercial exome panel, whose measured behavior on NA12878 is the best-quantified target in the file — Twist Exome 2.0 achieves 98.00% coverage uniformity at 30x depth with a fold-80 base penalty of 1.31, SNP precision above 99%, and IDT's panel shows INDEL precision above 94% and sensitivity above 89% (Performance comparison of four exome capture platforms on DNBSEQ-Series high throughput sequencer). This baseline defines what a mature, heavily-tuned panel achieves on coding targets; our method must match it there and extend coverage to regions it does not target at all.

The strongest existing method for the placement sub-problem is the tiling-array dynamic program with the bridge strategy from Design optimization methods for genomic DNA tiling arrays, which achieves roughly 90% coverage of non-repetitive base pairs while holding repeat inclusion to about 4%, a ~19–20% improvement over naive repeat-skipping. We will run its public tooling on our target loci, then append the best available sequence-level post-filter (thermodynamic screening with the constraint checks from Stable DNA Sequence Over Close-Ending and Pairing Sequences Constraint) so the comparison isolates placement strategy rather than thermodynamic awareness. Fairness conditions: identical target loci, identical total probe count and synthesis budget, identical depth in simulation and identical sequencing runs on the reference standards, and an equal tuning budget spent on each method's hyperparameters (grid search over the DP's scoring weights versus our penalty weights).

The strongest equal-effort competitor for the whole problem is a discrete stochastic search: genetic algorithm or the improved chaos-whale optimization of Stable DNA Sequence Over Close-Ending and Pairing Sequences Constraint over the same parameterization, evaluated with the same NUPACK-based thermodynamics and given wall-clock compute matched to our gradient method. This baseline matters because if it matches our coverage and constraint compliance, the differentiable machinery is not earning its complexity.

The key ablation removes the proposal's central ingredient: the same coverage objective and parameterization are optimized unconstrained, and violating probes are then filtered or trimmed post hoc, exactly as clinical panel pipelines do today. This isolates proactive constraint integration from representation choice. A second ablation replaces the continuous violability density $\rho(c)$ with a binary RepeatMasker mask (constraints enforced only outside repeats), isolating continuous bridging from constraint integration itself. Success for the proposal means the full method beats the post-hoc-filtered ablation on repeat-region coverage without an increase in end-of-design constraint violations, and beats the binary-mask ablation on dark-region base coverage at equal repeat inclusion.

Why not just…?

Why not just sequence the whole genome instead of designing better panels? Short-read genome sequencing adds only 5–20% diagnostic yield after a negative exome, spreads reads across three billion bases so depth at any given locus is shallow (roughly 30x versus hundreds-fold for targeted capture, which cripples detection of low-level mosaicism and GC-rich regulatory sequence), and the more advanced alternatives — long-read sequencing, optical genome mapping, untargeted RNA-seq — remain largely confined to the research setting with limited payor reimbursement (Beyond the exome: What's next in diagnostic testing for Mendelian conditions). Targeted, high-depth capture is therefore an indispensable clinical modality for the foreseeable future, and its design quality directly caps diagnostic yield. That said, this is a real limitation of the proposal's scope: for laboratories with routine long-read access, custom short-read panels add less. The PancancerLight 800 experiment (793 sites at 1–100% allele frequency) will quantify the regime — low-allele-frequency detection at high depth — where panel-based capture retains a decisive advantage.

Why not just post-filter a naive probe design? This is the exact hypothesis the core ablation tests, and the engineering literature predicts it fails: post-processing an unconstrained design to satisfy manufacturing rules destroys the optimized quantity, whether that is a photonic resonance (Machine learning to optimize additive manufacturing for visible photonics), lithographic edge placement (DiffOPC achieved an edge placement error of 2.2, a 19–71% reduction over post-processed pixel methods, while cutting mask shot count 62–81%: Differentiable Edge-based OPC), or structural validity (unconstrained GANs produced floating material in ~68% of designs, which proactive guidance cut to ~6%: Diffusion Models Beat GANs on Topology Optimization). The genomic version of the failure is documented too: naive repeat-skipping in tiling design cost ~19–20% coverage (Design optimization methods for genomic DNA tiling arrays). If the ablation shows post-filtering matches our method on repeat-region coverage and constraint compliance, the proposal's core claim is wrong and we will say so.

Why not just improve downstream variant interpretation instead of upstream capture? Because downstream methods cannot rescue a variant that was never read. Refined preferences of prioritizers improve intelligent diagnosis for Mendelian diseases raised Exomiser's top-1 rate to 61.6% in trio mode, and TriFit: Trimodal Fusion with Protein Dynamics for Mutation Fitness Prediction lifted mutation-fitness AUROC to 0.897 on ProteinGym, far above zero-shot ESM3's 0.769 — but both operate on variants that already passed through the capture panel and aligner. When a deep intronic causative variant sits in a region with no probes, no downstream scorer, however good, can rank it. The two directions are complementary, and this proposal deliberately targets the upstream step that no cited work addresses.

Why not just use discrete search (genetic algorithms, simulated annealing, whale optimization)? The one existing constraint-satisfying sequence designer, the improved chaos-whale method of Stable DNA Sequence Over Close-Ending and Pairing Sequences Constraint, works on small de novo pools for DNA computing; the tiling work of Design optimization methods for genomic DNA tiling arrays found exact combinatorial optimization infeasible for whole chromosomes and had to retreat to linear-time dynamic programming. Gradient methods with differentiable constraints are precisely what closed the gap at engineering scale — DiffOPC's 62–81% cost reduction over pixel-based search and Neuroshaper's demonstration that overparameterized continuous representations smooth the landscape past local minima that trap primitive-based optimizers (Shaping freeform nanophotonic devices with geometric neural parameterization). Still, this is a genuine threat at panel scale, which is why the equal-compute discrete-search baseline is run as a first-class competitor rather than dismissed: if it matches our uniformity and compliance metrics within noise, the differentiable machinery should be abandoned in favor of the simpler search.

Decisive Checks

Three results would settle the proposal's fate. First, on the ablation: constraint-aware design must beat post-hoc-filtered design on repeat-flanked and GC-extreme target coverage while ending with fewer constraint violations, not merely trading one for the other. Second, on wet-lab NA12878 and PancancerLight 800 runs: the designed panels must reproduce the simulator's predicted uniformity ordering across methods, and retain detection of the 793 reference sites at low allele frequencies at depth comparable to or better than the commercial coding-panel benchmarks (98.00% uniformity at 30x, fold-80 penalty 1.31). Third, on the clinical simulation: the CORD non-coding panel must show that the intronic and regulatory space of all 25 genes — currently invisible to coding-only exomes — is coverable at callable depth with a compliant, synthesizable probe library, converting a structural blind spot into a testable region. If the first check fails, the engineering analogy does not transfer to genomics and the proposal is refuted; if it succeeds, the diagnostic gap acquires its first genuinely design-side attack, and the large toolbox of constraint-integration methods — surrogate guidance, level-set bridging, velocity scaling — opens to a field that has never used it.