Scalable Gradients for Stochastic Differential Equations

Xuechen Li, Ting-Kam Leonard Wong, Ricky T. Q. Chen, David Duvenaud

Abstract

The adjoint sensitivity method scalably computes gradients of solutions to ordinary differential equations. We generalize this method to stochastic differential equations, allowing time-efficient and constant-memory computation of gradients with high-order adaptive solvers. Specifically, we derive a stochastic differential equation whose solution is the gradient, a memory-efficient algorithm for caching noise, and conditions under which numerical solutions converge. In addition, we combine our method with gradient-based stochastic variational inference for latent stochastic differential equations. We use our method to fit stochastic dynamics defined by neural networks, achieving competitive performance on a 50-dimensional motion capture dataset.

Introduction

Neural Ordinary Differential Equations (ODEs) have gained prominence due to the adjoint method, which enables training with constant memory regardless of depth. However, many real-world systems exhibit inherent stochasticity. While Stochastic Differential Equations (SDEs) are the natural framework for modeling such uncertainty, they present significant challenges for gradient computation. Existing approaches, such as pathwise differentiation or backpropagating through the solver, either scale poorly with parameter count or require prohibitive memory to store intermediate states. This work introduces the Stochastic Adjoint Sensitivity Method to bring the efficiency of ODE adjoints to the stochastic domain.

Methods

The proposed method addresses two primary hurdles: the mathematical difficulty of reversing SDEs and the computational cost of noise storage.

The Stochastic Adjoint and Stratonovich Formulation

Standard Itô calculus lacks a symmetric time-reversal property, making backward integration problematic. The authors demonstrate that by adopting the Stratonovich formulation, SDE dynamics become time-symmetric. This allows for a backward Stratonovich SDE that computes gradients by negating the drift and diffusion functions, providing a mathematically sound basis for the adjoint method.

Virtual Brownian Tree

A critical requirement for the adjoint method is using the exact same noise during the backward pass as was sampled during the forward pass. Storing this noise at every time step would lead to O(L) memory complexity. To solve this, the authors introduce the Virtual Brownian Tree.

The Virtual Brownian Tree reconstructs noise on the fly with O(1) memory.

By utilizing splittable random number generators, the solver can query the noise value at any arbitrary time t in O(log L) time while maintaining O(1) memory. This effectively transforms a storage bottleneck into a light computational task.

Experimental Evaluation

The method was evaluated on synthetic benchmarks and a high-dimensional 50-D motion capture (MoCap) dataset.

Efficiency and Accuracy

Numerical experiments confirmed that the adjoint gradients closely match analytical solutions, with error decreasing as the solver's step size is reduced.

Efficiency and accuracy comparison across different SDE solver methods.

As shown in the efficiency comparison, the method maintains high accuracy while offering significant efficiency gains. On the MoCap task, the Latent SDE framework outperformed Latent ODEs, achieving a Test MSE of 4.03 compared to 5.98, demonstrating the advantage of modeling multi-modal distributions in high-dimensional physical systems.

Conclusion

The Stochastic Adjoint Sensitivity Method enables the training of high-dimensional Neural SDEs with constant memory. By combining Stratonovich calculus with the Virtual Brownian Tree, this approach overcomes previous scalability limits. Future work includes extending the method to handle non-diagonal diffusion and exploring variance reduction techniques for gradient estimates.

Summary updated

Abstract

The adjoint sensitivity method scalably computes gradients of solutions to ordinary differential equations. We generalize this method to stochastic differential equations, allowing time-efficient and constant-memory computation of gradients with high-order adaptive solvers. Specifically, we derive a stochastic differential equation whose solution is the gradient, a memory-efficient algorithm for caching noise, and conditions under which numerical solutions converge. In addition, we combine our method with gradient-based stochastic variational inference for latent stochastic differential equations. We use our method to fit stochastic dynamics defined by neural networks, achieving competitive performance on a 50-dimensional motion capture dataset.
BibTeX
@inproceedings{li2020scalable,
  author = {Xuechen Li and Ting-Kam Leonard Wong and Ricky T. Q. Chen and David Duvenaud},
  title = {{Scalable Gradients for Stochastic Differential Equations}},
  booktitle = {International Conference on Artificial Intelligence and Statistics},
  year = {2020},
  url = {https://openreview.net/forum?id=lI_a4sdEUv},
}
Figure