Which Bases Drive Both Directions? Calibrated Mutagenesis Tests for Bidirectional Promoters

Many human genes share a promoter with a neighbor pointing the opposite way. A single stretch of DNA starts transcription in both directions, often producing a protein-coding gene on one side and a non-coding RNA on the other. Deep sequence models can now predict how much transcription starts on each DNA strand. This suggests a direct computational question: for each short sequence element inside such a promoter, does mutating it change output in both directions (a shared switch), mostly in one direction (a strand-specific element), or in opposite directions (a toggle)? The usual tool for this is in-silico saturation mutagenesis, which means computationally trying every possible single-base change and recording how the model's prediction moves. Current practice ranks the resulting effect scores and reads off the largest ones. It never asks whether those effects are larger than what a random, compositionally similar piece of promoter DNA would produce, or whether picking the top few out of thousands of windows has inflated them.

We propose a joint two-strand significance test for mutagenesis effects. Each window's pair of effects, one per strand, is compared against a null built from other windows whose GC content, CpG density, DNA shape profile, and position relative to both start sites match. The whole scan-and-select pipeline is then recalibrated on that null. The test adds almost no model evaluations, because the null draws come from mutagenesis results already computed at other promoters. It produces calibrated p-values and a three-way label (shared, strand-specific, toggle) for every element of every bidirectional promoter.

Core Claim

A composition-matched, locally permuted null for paired strand effects should give nominal false-positive rates where the two common shortcuts fail. The first shortcut is thresholding raw effect magnitude. The second is comparing against scrambled sequences that keep dinucleotide counts but lose positional context. The paired test should also find more shared switches than testing each strand separately and intersecting the calls. The claim is about model-level explanation, in the sense of Interpretable machine learning for genomics, which reviews attribution methods and warns that they explain the model, not the cell. We therefore treat biological validation as a separate, weaker check rather than the headline.

Research Program

The model is a BPNet-style convolutional network that predicts base-resolution, strand-specific transcription-start profiles. We train it on a strand-resolved start-site assay, using EPDnew human promoters for region selection. The mutagenesis engine is fastISM. That work speeds up mutagenesis by recomputing only the receptive field a mutation touches, and it reports being 1.6x–2.1x faster than standard mutagenesis on BPNet and over 50x faster than DeepSHAP or Integrated Gradients on that multi-output model. fastISM scores each output channel independently and computes no significance. Our contribution starts where it stops.

PromID: human promoter prediction by deep learning observes that promoters lacking a TATA box, which includes most bidirectional ones, show smeared rather than sharp start sites. For this reason we summarize each strand as total predicted log initiation in a window around its start site, not as a peak height. For a sequence window $w$, let $a_w$ and $b_w$ be the mean mutagenesis effects on forward-strand and reverse-strand initiation. We rotate these into a shared component and a differential component:

$$S_w = \frac{a_w + b_w}{\sqrt{2}}, \qquad D_w = \frac{a_w - b_w}{\sqrt{2}}.$$

Discovery uses the joint statistic $T_w = S_w^2/\hat\sigma_S^2(Z_w) + D_w^2/\hat\sigma_D^2(Z_w)$. Here $Z_w$ is the window's covariate vector (GC fraction, CpG observed/expected ratio, a dinucleotide structural profile of the kind used by A successful hybrid deep learning model aiming at promoter identification, distances to both start sites, and promoter-level predicted output). The two variances are estimated from null neighbors. The quadratic combination of two moment-like components follows A Powerful Statistical Method for Identifying Differentially Methylated Markers in Complex Diseases. That work combined mean and variance shifts in methylation and found 1,318 loci by permutation, against 750 with a t-test. We borrow the construction, applied to two strands rather than two moments.

The null borrows its method from K-Nearest-Neighbor Local Sampling Based Conditional Independence Testing. That test swaps values only among nearest neighbors in confounder space and reports an F-score of 0.82 against a next-best of 0.78 on flow cytometry data. Here the swapped objects are effect pairs $(a,b)$ from the $k$ nearest windows in $Z$-space, drawn from other promoters. The reasoning for recycling observed values rather than generating synthetic sequences comes from The conditional permutation test. There, permuting observed values kept Type I error near 5%, while synthetic conditional draws inflated it to 15–30% under misspecification. The null hypothesis is that a window's effect pair is exchangeable with those of composition-matched windows. This is a background-matching null, not a formal conditional-independence null, and we state that limitation openly.

Two further steps complete the pipeline. The first addresses search bias. Following The choice of null distributions for detecting gene-gene interactions in genome-wide association studies, in which an interaction with asymptotic $p = 4.226\times10^{-7}$ fell to $p = 0.148$ once the full screening pipeline was rerun under permutation, we rerun the entire scan, selection of top windows, and motif merging on null-substituted promoters. The corrected p-values come from that pipeline-wide null. The second step labels each significant window as shared, strand-specific, or toggle, based on the signs and relative sizes of $a_w$ and $b_w$ tested against the same null.

Baselines

The naive reference is raw mutagenesis magnitude: rank windows by $|a|+|b|$ and call the top 1% per promoter. Practitioners commonly do exactly this. Comparing against it shows how many calls a calibrated test removes.

The strongest competitor on the actual question is the standard genomics null: dinucleotide-shuffled copies of each promoter, scored with fastISM, with an empirical p-value computed separately for each strand. It is the strongest because it is already non-parametric, preserves composition, and is widely trusted. It will be given the same model, the same number of null draws per test, and the same multiple-testing correction. To count as beaten, it must show inflated false positives or lower power at matched false-positive rate on the planted-truth benchmark described below. We have no reported result on Lacuna for calibrated strand-paired mutagenesis significance, so we cannot quote a target number. The only relevant published figures are the CPT and k-NN calibration results above, from non-genomic data. For a sanity check that the underlying model is competent, a promoter/non-promoter classification head should approach PromID's reported MCC of 0.76 on EPDnew, noting that HMPI reports 0.8151 on its own human dataset.

A gradient baseline runs DeepSHAP on each strand output with the same null and correction. It tests whether perturbation-based effects matter at all.

Three ablations isolate each ingredient. The first uses uniform permutation over all windows, removing $Z$-conditioning. The second drops pipeline-wide recalibration. The third tests each strand separately and intersects the calls, removing the joint statistic.

Why not just…?

Why not just use dinucleotide-shuffled sequences as the null? This is the most serious alternative, and it could come close in calibration. Shuffling keeps base-pair composition but erases distance to each start site and flanking motif context. In bidirectional promoters that context is strongly structured, much as nested structure inflated nulls in Multivariate classification of neuroimaging data with nested subclasses, where the correct null mean reached 78.5%. If the planted-truth experiment shows shuffling is calibrated and equally powerful, the $Z$-matching step adds nothing, and we would report the pipeline-wide correction and the paired statistic as the contributions.

Why not just test each strand separately and intersect? Intersecting two separate tests is valid but conservative. A shared switch with moderate effects on both strands can fail both tests individually while standing out on the joint statistic. The intersection ablation measures this power gap directly. If the gap is negligible, we will simplify to the intersection.

Why not just use gradients, which are cheaper? On BPNet, fastISM reports gradient methods taking over 50x longer because they need a backward pass per output coordinate. FANS (Feature Attribution with Necessity and Sufficiency via Dual-stage Perturbation Test) reports 26–47% lower infidelity than Integrated Gradients, though on images. Gradient baselines run inside our pipeline will show whether they lose power on sequences as well.

Proposed Experiments

The primary experiment is a planted-truth benchmark. We take real bidirectional promoter backgrounds and implant motif instances whose effects on the trained model are known and designed to be shared, strand-specific, toggle, or null. We also generate fully null promoters by substituting matched windows. On null promoters, the decisive check is whether each method's false-positive rate at a 5% threshold stays nominal across GC strata. On planted promoters, we measure power and the accuracy of the three-way label.

The second check uses orientation. Reverse-complementing a promoter swaps the roles of the two strands. A coherent test must return mirrored labels with matching p-values, and we report the disagreement rate for every method.

The third check touches biology while acknowledging Watson's caution. Applied to the promoter shared by SLC12A2 and LINC01184, the method should label the window containing rs3812049 as a shared switch. In Genome-wide association study of red blood cell traits in Hispanics/Latinos, CRISPR disruption of this element reduced both transcripts. We also examine the ENCODE-pilot promoters from Integrated analysis of experimental data sets reveals many novel promoters in 1% of the human genome, where 76% of tested novel regions were confirmed by 5'-RACE and 25% were active in transfection. Regions validated by transfection should contain more significant windows than unvalidated ones. This is a correlational check only. A failure here would not falsify the model-level claim, but a strong pattern would suggest the calls track real regulatory elements.