Tri-Scale Neural ODEs for Continuous Multi-Omics Disease Modeling

Shohaib Shaffiey, Massimiliano PierobonICML 2026 · OpenReview

Introduction

Biological systems are inherently multi-scale, with processes evolving at vastly different speeds. Gene transcription occurs over hours, protein synthesis takes minutes, and metabolic reactions happen in seconds. In mathematical terms, a system where variables change at such disparate rates is known as a stiff system.

When researchers attempt to model these dynamics using standard Neural Ordinary Differential Equations (NODEs), they encounter significant hurdles. Standard solvers either require infinitesimally small time steps to remain stable—rendering them computationally infeasible—or the model "cheats" by ignoring fast-moving variables to achieve a low error score, a phenomenon known as over-smoothing. This limitation makes it impossible to accurately predict how a cell might respond to a drug that acts within a narrow, fast-moving time window.

To address this, researchers have introduced Tri-Scale Neural ODEs, a hierarchical framework designed to model biological systems across different temporal scales. By integrating biological hierarchy and stability constraints directly into the architecture, this approach successfully models stiff dynamical systems that typically cause standard AI models to crash or over-smooth critical data.

Methodology: The Tri-Scale Stiff NODE

The proposed Tri-Scale Stiff NODE mirrors the "Central Dogma" of biology (DNA $\rightarrow$ RNA $\rightarrow$ Protein $\rightarrow$ Metabolites). Key innovations of this architecture include:

  • Hierarchical Latent Spaces: Instead of relying on a single vector, the model utilizes three coupled latent spaces (Gene, Protein, and Metabolite) with an asymmetric causal flow.
  • Learnable Timescales: The model learns specific time constants ($\tau_g, \tau_p, \tau_m$) for each layer, enforcing the biological reality that $\tau_{gene} < \tau_{protein} < \tau_{metabolite}$.
  • Lipschitz-Constrained Damping: To prevent numerical divergence (the system "blowing up"), linear damping terms are added. The authors provide a mathematical proof using Lyapunov analysis to guarantee system stability even when the stiffness ratio exceeds $10^6$.
  • Mutual Information Regularization: A Neural Information Estimator (MINE) ensures that fast-scale metabolite dynamics remain statistically coupled to slower regulatory gene dynamics, preventing the layers from decoupling during training.

Datasets and Validation

The model was validated on two primary longitudinal multi-omics datasets:

  1. STATegra B-cell Dataset: A comprehensive dataset measuring gene expression (12,762 transcripts), protein abundance (2,654 markers), and metabolites (44 species) during B-cell differentiation.
  2. Traxler Macrophage Dataset: A dual-omics dataset (ATAC-seq and RNA-seq) demonstrating the model's ability to generalize across different biological scales (chromatin to transcription).

Additionally, the LINCS L1000 Library (over 22,000 compounds) was utilized for a proof-of-concept drug repurposing pipeline.

Key Insights and Results

  • Architecture as a Solver: Rather than relying on expensive implicit numerical solvers (which scale at $O(d^3)$), stiffness is resolved at the architectural level. This enables the use of efficient explicit solvers like Dopri5.
  • Stiffness Mitigation: The Tri-Scale architecture successfully reduced the effective stiffness ratio from a raw $1.3 \times 10^6$ down to $1.61 \times 10^3$, making the system computationally tractable.
  • Stability vs. Accuracy: An ablation study revealed that removing damping terms decreased training error for genes but caused a catastrophic 9,839% increase in error for metabolites. This underscores that low training error in stiff systems can often indicate biological invalidity (over-smoothing).
  • Superior Generalization and Interpolation: Because the model learns a continuous vector field, it can predict cellular states at unmeasured timepoints. On held-out timepoints, the Tri-Scale NODE achieved significantly lower Mean Squared Error (MSE) than baselines like scNODE (which over-smoothed) and HBNODE (which diverged).
  • Drug Repurposing: The model screened 22,268 compounds and identified 510 potential hits. Top candidates, such as Brefeldin A (affecting protein transport) and Tozasertib (a kinase inhibitor), were identified as biologically relevant to the B-cell differentiation manifold.

Limitations and Future Work

While highly effective, the current architecture is limited to three molecular scales, though it could theoretically be generalized to a $K$-layer cascade. Furthermore, the drug screening process relies heavily on transcriptomic signatures, which may miss drugs that act directly on metabolites without leaving a gene expression "footprint."

The authors also note that while the model identifies strong statistical couplings, these predictions require interventional validation (e.g., CRISPR experiments) before clinical application. Finally, the model currently assumes smooth dynamics and does not yet account for discrete "transcriptional bursts" or the stochastic noise inherent in single-cell biology.

Summary updated

Abstract

The fields of AI-based disease fingerprinting, drug discovery and repurposing are currently among the emerging frontiers of machine learning applied to medicine. One major challenge is to obtain robust $\textit{in-silico}$ modeling of disease progression while accounting for the vastly different time scales of biochemical interactions, from gene expression to protein abundance and metabolic flux. Discrete sequence models inadequately represent such multi-scale interactions, and standard Neural Ordinary Differential Equations (NODEs) often fail to train stably under stiffness (different time scales). To address this, a Tri-Scale Stiff NODE, defined by hierarchically coupled latent differential equations that model the causal relationships from genes to proteins and metabolites, is introduced and optimized in this paper in terms of reconstruction error and information-theoretic mutual information. This enables continuous-time modeling of cellular responses to identify not only the disease dynamics, but also drug perturbations that act within narrow time windows, often invisible to discrete-time approaches. Lyapunov analysis provides a theoretical guarantee that the modeled trajectories remain stable and well-behaved even under extreme stiffness. The methodology is validated using the STATegra B-cell and Traxler macrophage datasets, with the former utilized for a proof-of-concept drug repurposing pipeline.
BibTeX
@inproceedings{shaffiey2026tri,
  author = {Shohaib Shaffiey and Massimiliano Pierobon},
  title = {{Tri-Scale Neural ODEs for Continuous Multi-Omics Disease Modeling}},
  booktitle = {International Conference on Machine Learning},
  year = {2026},
  url = {https://openreview.net/forum?id=PsjSZaunzO},
}
Figure