Two-Bank Convolutional Motif Scanners for Cross-Species Transcription Factor Binding

Models that predict where a transcription factor (a protein that switches genes on or off by binding short DNA patterns) attaches to the genome are usually convolutional networks. Their first-layer filters act as learned motif scanners: each filter slides along the DNA and fires wherever the sequence matches a short preferred pattern. The trouble is that the filters do not care whether the pattern is a genuine binding site or a feature of the background genome. Examples of such background features are a repetitive element common in one species, such as the human Alu repeat, or a regional shift in GC content. A model trained on human data learns some of both, and it degrades when applied to mouse, whose repeats and composition differ. We propose to split the first convolutional layer into two filter banks. A conserved bank must explain binding in a way that is statistically independent of which species the sequence comes from, given its binding label. A species bank is explicitly encouraged to absorb species identity. At transfer time the species bank's contribution is simply removed.

The idea brings latent-subspace disentanglement from the fairness literature into convolutional motif detection, at the level where confounding actually enters these models: the filters themselves. The confounder here is not a scalar such as "GC fraction" that a covariate could absorb. It is motif-shaped. A repeat element is detected by exactly the same machinery as a binding site, so it can only be separated by giving it its own filters. If the approach works, it should give better cross-species prediction than distribution alignment. It should also produce a byproduct that alignment cannot: a set of filters that a biologist can read as "what this model considered species background," checkable against repeat annotations.

Core Claim

The closest existing work is MORALE, from "Frustratingly easy" domain adaptation for cross-species transcription factor binding prediction. It matches the mean and covariance of a single shared embedding between human and mouse, and it beats both a naive model and an adversarial gradient-reversal model on human-to-mouse transfer. Alignment of this kind forces the whole representation to look the same across species, so any species-varying feature is either erased or smeared across all dimensions. MORALE therefore cannot say which features were species background, and it cannot keep such features for within-species prediction while discarding them for transfer.

Our claim is narrower and testable. Give species background a designated set of filters, and enforce conditional independence between the conserved filters and species. The conserved filters then (a) transfer at least as well as MORALE to a held-out species, (b) recover the target factor's canonical motif with higher fidelity, and (c) leave the species filters matching annotated repeats far more often than chance. If (a) fails while (b) and (c) hold, the method is still a better interpretability tool, but not a better transfer method. We will report it as such.

Research Program

The model has a first convolutional layer with $K$ filters, split into a conserved bank of $K_c$ filters and a species bank of $K_s$ filters. Each bank feeds its own small pooling-and-dense pathway, and their outputs are added as logits. Write $f_c(x)$ and $f_s(x)$ for the pooled features of the two banks on sequence $x$, $y$ for the binding label, and $s$ for the species label. Training minimizes

$$ \mathcal{L} = \ell\big(y,\; g_c(f_c(x)) + g_s(f_s(x))\big) + \lambda_1\, \ell_{\text{sp}}\big(s,\; h(f_s(x))\big) + \lambda_2 \sum_{y'} \mathrm{HSIC}\big(f_c(x),\, s \mid y = y'\big), $$

where $\ell$ is the binding loss, $g_c$ and $g_s$ are the two binding heads, $h$ is a species classifier on the species bank, and HSIC is a kernel measure of statistical dependence computed separately within bound and unbound sequences. Conditioning on $y$ matters. Unconditional invariance would also strip away genuine species differences in how often a factor binds, which is the kind of utility loss the fairness literature warns about in Flexibly Fair Representation Learning by Disentanglement and Disentangled Representation with Causal Constraints for Counterfactual Fairness.

The species term needs only species labels, not binding labels. Unlabeled target-species sequences can therefore train $h$, which gives our method the same data access MORALE uses. At test time on a new species we predict with $g_c(f_c(x))$ alone. This is the "digital surgery" of FFVAE: zeroing designated dimensions at inference without needing the sensitive attribute.

Each design choice borrows from a specific prior paper and changes it. From FFVAE we take the idea of a designated sensitive subspace that can be masked at test time. FFVAE builds that subspace inside a variational autoencoder over images and tabular data; we build it in the filter bank of a discriminative scanner and drop the generative decoder, because reconstructing DNA is costly and is not what we need. From Learning Bias-Invariant Representation by Cross-Sample Mutual Information Minimization (CSAD) we take the observation that a reserved bias subspace is not enough, because information leaks across the partition unless dependence is explicitly penalized. We use a closed-form kernel penalty rather than CSAD's mutual-information estimator and graph random walks, in the same spirit as MORALE's preference for closed-form objectives over adversarial training.

Two papers motivate rather than supply the method. Learning In-Silico Maps of Transcription Factor Binding and Cooperativity interactions shows that convolutional binding models often rely on cofactor and pioneer-factor motifs rather than the target factor's own motif. Deepprune shows that wide convolutional layers devote many "auxiliary kernels" to background. Both diagnose the problem; neither separates the signals. We deliberately do not try to separate cofactors in this proposal. Cofactor binding is often conserved and causally real, so there is no clean label like species to anchor it. Species background is the case where the confounder has cheap supervision and a falsifiable biological identity.

Baselines

The naive reference is a single-bank convolutional network of the MORALE architecture trained on human data only and applied directly to mouse. MORALE reports an average auPRC of 0.31 for this model across CTCF, HNF4α, RXRA and CEBPA in human-to-mouse transfer. Anything that fails to beat this number is not doing domain adaptation at all.

The strongest existing method is MORALE itself, at an average auPRC of 0.332 on the same task. Its adversarial gradient-reversal comparison reaches 0.304, as reported in "Frustratingly easy" domain adaptation for cross-species transcription factor binding prediction. MORALE is the right competitor because it addresses exactly this confounder with the same compact architecture of roughly 613k parameters and the same data access. To keep the comparison fair, we reimplement it on identical train, validation and test chromosome splits. Both methods get the same unlabeled mouse sequences, the same total filter count $K$, and the same hyperparameter search budget, measured as a fixed number of training runs selected on held-out human chromosomes only. No target labels are used for selection. Beating MORALE means a higher mean auPRC across the four factors, with the paired per-factor difference significant over at least five random seeds.

In the five-species setting, MORALE reports about 0.35 auPRC on human CEBPA when trained on several species, against about 0.22 for human-only training. We run a leave-one-species-out version in which our method trains its species bank on the four training species, and we compare against both pooled multi-species training and MORALE.

Two further baselines are simple preprocessing and pruning. The preprocessing baseline is a single-bank model trained on repeat-masked input with GC-matched negative sets. The pruning baseline is a Deepprune-style model reduced to a small number of kernels, following Deepprune, which reports beating unpruned networks on 77.1% of 690 ENCODE datasets; no cross-species result was found on Lacuna for it.

Three ablations isolate the key ingredient. The first keeps two banks and additive heads but sets $\lambda_2 = 0$, so that only the architectural split and the species classifier remain. The second keeps the full objective but makes the HSIC term unconditional. The third uses a single bank with species-specific output heads, which tests whether a shared trunk with per-species readouts already achieves the effect. The claim rests on the full model beating the first ablation. If two banks without the dependence penalty transfer just as well, then the disentanglement contributes nothing, and the result is merely a structured multi-task model.

Why not just…?

Why not just mask repeats and match GC content in the training data? This is the most serious threat for the repeat-and-composition confounder specifically, and we run it as a baseline rather than argue it away. Two reasons suggest it will fall short. First, repeat masking removes only annotated repeats, while species background also includes diverged, partially annotated elements and k-mer composition. Second, On the Fairness of Disentangled Representations shows that deep networks reconstruct sensitive factors from correlated input features even when the explicit attribute is withheld, and Disentangled Representation with Causal Constraints for Counterfactual Fairness shows that blinding the input costs utility when the removed content carries legitimate signal. Real binding sites do occur inside repeats. If the masked baseline matches our transfer auPRC, we will conclude that the confounder in this benchmark is adequately described by annotated repeats plus GC content. We would then reposition the method around the cases masking cannot reach, namely its auPRC on binding sites that lie inside repeats.

Why not just use MORALE's moment alignment? Alignment is cheaper and already works, so the burden is on us. Alignment matches the aggregate statistics of one shared embedding. It therefore cannot tell us which features were species background, and conditional differences in binding between species are pushed into the same alignment pressure. The decisive measurements are the species-decodability probe on the conserved features and the repeat-overlap analysis described below, neither of which MORALE can produce. If our transfer auPRC only ties MORALE's, the contribution reduces to interpretability, and we will say so.

Why not just pool more species into training? MORALE's own five-species result, about 0.35 versus 0.22 for CEBPA, shows that more species diversity alone helps a great deal, so this is a real competitor. Pooling, however, gives the network no reason to keep species-specific filters out of the features it uses for prediction. It only dilutes any one species' background. The leave-one-species-out comparison against pooled training with an identical filter budget settles this question directly.

Why not just interpret a standard model post hoc and discard the bad filters? Learning In-Silico Maps of Transcription Factor Binding and Cooperativity interactions shows that attribution reveals entanglement without removing it. Deepprune-style pruning deletes filters based on how much they matter to the prediction, not on whether they track species. A filter that is both predictive and species-specific survives pruning. Our pruning baseline tests whether this distinction matters in practice.

Proposed Experiments

The primary benchmark is the multi-species ChIP-seq transfer benchmark used by MORALE, covering human, macaque, mouse, rat and dog for CTCF, HNF4α, RXRA and CEBPA. We run two protocols: the two-species human-to-mouse protocol with auPRC on held-out mouse chromosomes, and the leave-one-species-out protocol. As a within-species sanity check we train the same architecture on a sample of the 690 ENCODE ChIP-seq datasets used by Deepprune. There we require that the two-bank model's in-distribution AUC not fall below the single-bank model's. A method that buys transfer by losing in-species accuracy is the fairness-utility trade-off in a new setting.

Three checks are decisive beyond auPRC. The first is a leakage probe: a linear classifier trained to predict species from $f_c$ should be near chance under the full model and clearly above chance under the $\lambda_2 = 0$ ablation and the single-bank model. The second is motif fidelity: following MORALE's own evaluation, we run TF-MoDISCo on attributions from $g_c(f_c)$ and compare recovered motifs to JASPAR with Tomtom, expecting stronger matches to the canonical motif than either the naive model or MORALE.

The third is the falsifiable biological check. Species-bank filters, converted to position weight matrices, should match repeat-family consensus sequences, and their top activations should overlap repeat annotations at well above the genomic base rate. If instead the species bank fills with GC-rich k-mers and nothing repeat-like, the method has separated composition but not repeats, which is a weaker result. If it fills with the target factor's own motif, the independence penalty has failed, and the method does not work as claimed.

Finally, we stratify mouse test peaks by whether they fall inside B1 or B2 repeats, the rodent counterparts of Alu. The prediction is that our gains over MORALE concentrate in the repeat-overlapping stratum. A gain spread uniformly would suggest an ordinary regularization effect rather than disentanglement.