TL;DR
This paper provides the first provable identifiability bounds for learning stochastic differential equation (SDE) parameters solely from stationary (steady-state) distributions under shift interventions, without requiring temporal trajectory data. The authors show that linear low-rank SDEs require roughly $r$ interventions (where $r$ is the hidden latent rank), while nonlinear SDEs in the small-noise regime are identifiable with $r+1$ interventions even when the elementwise activation function is unknown.
What Problem Does This Paper Address?
In biology and physics, dynamical systems are often modeled via SDEs, but real-world measurements are frequently destructive (e.g., single-cell RNA sequencing with CRISPR perturbations). As a result, researchers cannot track cells over time—they only observe the system's stationary distribution after perturbation.
While classical causal discovery relies on Directed Acyclic Graphs (DAGs), DAGs struggle with cyclic feedback loops and continuous dynamics. Conversely, fitting SDEs directly usually requires full time-series trajectories or infinitesimal generators. This leaves open a critical theoretical question: How many distinct interventions are necessary to uniquely identify an SDE's governing parameters when we can only observe its stationary distribution?
Methods
The authors analyze low-rank parameterizations of the drift function where an ambient dimension $n$ projects onto a much smaller latent dimension $r$ ($n > 2r$):
- Linear SDEs: Drift is parameterized as $v(x) = (AB - D)x$, where $A \in \mathbb{R}^{n \times r}$, $B \in \mathbb{R}^{r \times n}$, and $D$ is a diagonal decay matrix. The stationary distribution is Gaussian, characterized by the Lyapunov equation.
- Nonlinear SDEs: Drift is modeled as $v(x) = A\sigma(Bx) - x$, where $\sigma$ is an elementwise contractive activation. The authors employ SDE perturbation theory to analyze the small-noise regime ($\epsilon \to 0$), showing that stationary means and rescaled covariances approximate a linearized system at each intervention's fixed point.
- Learnable Activations: Motivated by the finding that exact knowledge of $\sigma$ is unnecessary for identifiability, the authors parametrize the drift using low-rank matrices combined with learnable Multi-Layer Perceptrons (MLPs) as elementwise activation functions, trained via moment matching, Kernel Deviation from Stationarity (KDS), or Sinkhorn divergence.
Datasets
The authors evaluated their methods on synthetic and semi-synthetic benchmark setups:
- Synthetic Linear SDEs: Randomly generated systems ($n=100, 200$; rank $r \in \{2, 4\}$) under random Gaussian shift interventions.
- Synthetic Nonlinear SDEs: SDEs governed by sinusoidal/cosine activations with small noise ($\epsilon = 10^{-5}$ for recovery, $\epsilon \in [0.05, 0.3]$ for generalization).
- Semi-Synthetic Gene Regulatory Networks (BoolODE): Simulated single-cell RNA-seq datasets using the BEELINE framework, which generates biologically realistic transcriptional/translational dynamics and cyclic networks with overexpression (CRISPR-a-like) interventions.
Key Insights and Ideas
- Noiseless limits hide structure: Counterintuitively, setting noise to zero ($\epsilon = 0$, reducing to deterministic ODE steady states) ruins low-rank identifiability, requiring $\Omega(n)$ interventions. Keeping stochasticity (even infinitesimal noise) provides second-moment covariance information via the Lyapunov equation, slashing the requirement down to order $r$.
- Tight Linear Recovery: For linear SDEs with known decay $D$, $r$ random interventions are sufficient to uniquely identify the drift almost surely, whereas at most $r-2$ interventions render the system unidentifiable.
- Model Invariance to Activation Functions: In the nonlinear low-noise limit, $r+1$ interventions suffice to identify $A$ and $B$ (up to column permutation and scaling) without needing to know the true activation function $\sigma$ beforehand.
- Flexibility via Neural Activations: Because the underlying activation does not need to be pre-specified, one can substitute rigid analytical functions with flexible, learned MLP activations without losing causal identifiability.
Key Results
- Linear Drift Recovery: With known decay, error drops to near-zero with $k = r$ interventions, whereas $k = r-2$ fails completely with high error and variance. When decay $D$ is unknown, an oversampling of $k = r \log(n)$ reliably recovers the drift.
- Nonlinear Parameter Recovery: Under small noise ($\epsilon = 10^{-5}$), moving from $k = r$ to $k = r+1$ interventions halves normalized parameter error (from $0.13 \to 0.065$ for $A$, and $0.14 \to 0.07$ for $B$) while drastically cutting variance across runs.
- Generalization to Unseen Interventions: In synthetic nonlinear generalization tests ($n=20, r=3$), learned MLP activations significantly outperformed fixed sigmoids at low noise (e.g., at $\epsilon = 0.05$, mean test distributional distance was 9.51 for MLP vs. 21.78 for sigmoid).
- Gene Regulatory Network (GRN) Discovery: On BoolODE benchmarks, the nonlinear SDE with a learned activation achieved the highest Area Under the Precision-Recall Curve (AUPRC $\approx 0.065\text{--}0.070$), beating both fixed-activation SDEs and the linear baseline (Bicycle).
Limitations and Future Work
- Small-Noise Assumption: The nonlinear identifiability guarantees strictly hold in the limit of infinitesimal noise ($\epsilon \to 0$); theoretical bounds for moderate or large noise remain an open problem.
- Contractive Dynamics: The theory currently assumes contractive drift (ensuring a unique global stable equilibrium per intervention), excluding systems with multi-attractor or chaotic landscapes.
- Observed Decay / Known Interventions: Linear proofs assume observed decay $D$ (though empirically learning diagonal $D$ worked), and intervention targets and magnitudes must be known.
- Future Directions: Incorporating sparse time-series trajectory snapshots alongside stationary distributions, expanding drift capacity beyond two-layer architectures, and handling unknown intervention values.
Summary updated