Scalable Gradients for Stochastic Differential Equations

Xuechen Li, Ting-Kam Leonard Wong, Ricky T. Q. Chen, David Duvenaud

Abstract

The adjoint sensitivity method scalably computes gradients of solutions to ordinary differential equations. We generalize this method to stochastic differential equations, allowing time-efficient and memory-efficient training of SDEs with millions of parameters. By combining a novel backward-time SDE for gradients with a "virtual Brownian tree" for memory-efficient noise reconstruction, we enable high-performance, continuous-time generative modeling of complex, noisy datasets.

Introduction

Neural Ordinary Differential Equations (Neural ODEs) have transformed continuous-time modeling by treating neural network layers as infinitesimal steps in an ODE. However, ODEs are inherently deterministic and cannot naturally account for the stochasticity found in real-world systems like financial markets, molecular dynamics, or sensor data. Stochastic Differential Equations (SDEs) provide the necessary mathematical framework for modeling such noise, but training them at scale has historically been a "pick your poison" scenario: either accept poor time-scaling with the number of parameters or suffer massive memory consumption by storing every intermediate simulation step.

The Stochastic Adjoint Sensitivity Method

To overcome these scaling limitations, this work introduces two primary innovations: the stochastic adjoint process and the virtual Brownian tree.

Symmetry through Stratonovich Calculus

A key challenge in computing gradients for SDEs is the need to "run the dynamics backward." While Itô calculus is standard in many applications, it does not obey the classical chain rule, making it difficult to reconstruct gradients without storing the entire forward trajectory. By shifting to Stratonovich calculus, the authors leverage its mathematical symmetry. Stratonovich integrals follow standard calculus rules, allowing a backward-time SDE to correctly reconstruct the gradient and match the forward path.

Reconstructing the forward trajectory using Stratonovich vs. Itô calculus. Stratonovich (blue) accurately matches the true solution when reversed, whereas Itô (red) drifts.

The Virtual Brownian Tree

Computing gradients in the backward pass requires access to the exact noise (Brownian motion) sampled during the forward pass. Storing this noise at every time step would lead to linear memory complexity relative to the number of steps. The Virtual Brownian Tree solves this by using a deterministic tree of random seeds and "Brownian bridges." This structure allows the model to re-generate any specific point of the noise on-the-fly, achieving $O(1)$ memory usage with only a minor logarithmic increase in computation time.

The Virtual Brownian Tree enables O(1) memory noise reconstruction by using a hierarchy of random seeds and Brownian bridges to sample the same noise path in both forward and backward passes.

Experimental Evaluation

The authors evaluated their method on several tasks, including synthetic benchmarks like the 3D Stochastic Lorenz attractor and high-dimensional real-world data. On a 50-dimensional Motion Capture (Mocap) dataset consisting of human walking sequences, the Latent SDE model achieved a Test MSE of 4.03, significantly outperforming Latent ODEs (5.98) and other state-of-the-art models like ODE2VAE (8.09).

Crucially, the stochastic adjoint method maintains constant memory complexity, whereas standard backpropagation through a solver scales linearly with the number of time steps. This allows for the training of SDEs with millions of parameters on standard hardware, enabling the modeling of multi-modal distributions where a system might evolve in several different directions from the same starting state.

Conclusion and Future Work

While this method represents a significant leap in scalable SDE training, some limitations remain. The current implementation is optimized for diagonal noise; general, non-diagonal diffusion requires approximating "Lévy areas," which is computationally expensive. Additionally, SDE solvers are generally more demanding than ODE solvers, and the derivation assumes smooth drift and diffusion functions, which may constrain certain neural network architectures (e.g., those using ReLU activations). Despite these challenges, the stochastic adjoint method opens new doors for high-performance generative modeling of stochastic systems.

Summary updated

Abstract

The adjoint sensitivity method scalably computes gradients of solutions to ordinary differential equations. We generalize this method to stochastic differential equations, allowing time-efficient a...
BibTeX
@inproceedings{li2020scalable,
  author = {Xuechen Li and Ting-Kam Leonard Wong and Ricky T. Q. Chen and David Duvenaud},
  title = {{Scalable Gradients for Stochastic Differential Equations}},
  booktitle = {International Conference on Artificial Intelligence and Statistics},
  year = {2020},
  url = {https://openreview.net/forum?id=lI_a4sdEUv},
}
Figure