Abstract
Understanding how DNA sequence encodes gene regulation remains a central challenge in genomics. While deep learning models can predict regulatory activity from sequence with high accuracy, their generalizability across species—and thus their ability to capture fundamental biological principles—remains limited. Cross-species prediction provides a powerful test of model robustness and offers a window into conserved regulatory logic, but effectively bridging species-specific genomic differences remains a major barrier. We present MORALE, a novel and scalable domain adaptation framework that significantly advances cross-species prediction of transcription factor (TF) binding. By aligning statistical moments of sequence embeddings across species, MORALE enables deep learning models to learn species-invariant regulatory features without requiring adversarial training or complex architectures. Applied to multi-species TF ChIP-seq datasets, MORALE achieves state-of-the-art performance—outperforming both baseline and adversarial approaches across all TFs—while preserving model interpretability and recovering canonical motifs with greater precision. In the five-species transfer setting, MORALE not only improves human prediction accuracy beyond human-only training but also reveals regulatory features conserved across mammals. These results highlight the potential of simple yet powerful domain adaptation techniques to drive generalization and discovery in regulatory genomics. Crucially, MORALE is architecture-agnostic and can be seamlessly integrated into any embedding-based sequence model.
Introduction
Deep learning has revolutionized our ability to predict where transcription factors (TFs) will bind to DNA. However, these models often suffer from a "domain" problem: a model trained on human data frequently fails when applied to a mouse, and vice versa. While the fundamental biochemical rules of TF binding are often conserved across millions of years of evolution, the "background noise" of the genome—such as repetitive elements and varying GC content—changes rapidly. Standard models often overfit to these species-specific quirks rather than learning the universal regulatory grammar. Bridging this gap usually requires complex adversarial training, which is notoriously difficult to tune and can sometimes degrade performance.
The MORALE Framework
To address these limitations, we introduce MORALE (Moment Alignment for Regulatory Logic Evaluation). Instead of using a complex adversarial branch to hide species identity from the model, MORALE employs a much simpler approach based on statistical alignment.
The framework operates in three main steps:
- Embedding Layer: DNA sequences are processed through a standard architecture (e.g., Convolutions and Recurrent layers) to create a latent representation or embedding.
- Moment Alignment: The model calculates the first moment (mean) and second moment (covariance) of these embeddings for both the source species (e.g., mouse) and the target species (e.g., human).
- Loss Function: A penalty is added to the training process if the statistical moments of the two species do not match. This forces the model to find a "species-invariant" way to represent DNA.
This approach is "frustratingly easy" because it requires zero extra parameters and can be expressed in a closed mathematical form. In contrast, adversarial methods like Gradient Reversal Layers (GRL) can add millions of parameters; in our tests, the GRL approach ballooned the model to over 9.4 million parameters, while MORALE maintained a compact size of approximately 613k parameters.
Key Results and Insights
Superior Cross-Species Performance
In a two-species human-to-mouse transfer task, MORALE achieved an average auPRC (Area Under Precision-Recall Curve) of 0.332, significantly outperforming the adversarial GRL (0.304) and the baseline (0.31). Crucially, while GRL sometimes caused performance to drop below the baseline, MORALE remained stable across all tested TFs (CTCF, HNF4α, RXRA, and CEBPA).
The "More is Better" Effect
In a five-species test involving humans, macaques, mice, rats, and dogs, a model trained with MORALE actually outperformed a model trained only on human data when predicting human binding sites. For the TF CEBPA, MORALE reached an auPRC of ~0.35 on human data, compared to ~0.22 for the human-only model. This suggests that by looking across many species, the model learns a "cleaner" version of the regulatory code that is not as easily visible in a single genome.
Enhanced Interpretability and Robustness
Using feature attribution (TF-MoDISCo), we found that MORALE discovered DNA motifs that were much closer to ground-truth biological motifs than those found by adversarial models. Furthermore, MORALE was particularly effective at handling species-specific repeats (like Alus in humans), which are typically a major source of error in cross-species transfer.
Conclusion
MORALE demonstrates that simple moment alignment is a powerful tool for building generalizable models in regulatory genomics. By focusing on species-invariant features, it achieves state-of-the-art results without the complexity of adversarial training. Future work will explore integrating MORALE into larger transformer-based foundation models and applying it to more complex, tissue-specific binding patterns.
Summary updated