Score-based neural ordinary differential equations for reaction-diffusions

Jianguo Hou, Wuchen Li

TL;DR

This paper introduces a Lagrangian particle method for solving reaction-diffusion-advection equations where total mass is not conserved. By equipping particles with "physical weights" and local density information, the authors use neural ODEs to accurately model complex systems—like spreading epidemics or growing tumors—where new populations emerge from regions that were initially empty.

What Problem Does This Paper Address?

Reaction-diffusion-advection (RDA) equations are the mathematical workhorses of biology and ecology, describing how densities (of cells, animals, or chemicals) move, spread, and—crucially—grow or shrink.

Traditional solvers face a "pick your poison" scenario:

  1. Grid-based methods: These work great in 1D or 2D but become computationally impossible in high dimensions (the "curse of dimensionality").
  2. Particle methods: These scale better to high dimensions but usually assume mass conservation. In a standard "probability flow," particles just move around; they don't multiply.

The specific challenge here is modeling non-conservative mass. If a population starts in a tiny corner and eventually fills a whole room, a standard particle method won't have enough particles in the "empty" space to represent the growth accurately. Furthermore, calculating the "score" (the gradient of the log-density) usually requires noisy estimations from neighboring particles.

Methods

The authors propose a Lagrangian method where each particle is more than just a point in space—it carries a "backpack" of information.

1. The Characteristic System

Each particle tracks four variables via a system of Ordinary Differential Equations (ODEs):

  • Position ($X_t$): Where the particle is.
  • Log-density ($\ell_t$): The local "thickness" of the population.
  • Score ($S_t$): The gradient of the log-density ($\nabla \log \rho$).
  • Weight ($\lambda_t$): A value tracking the change in physical mass.

By evolving the score directly along the particle's trajectory, they avoid the noisy step of estimating it from a cloud of neighbors.

2. Self-Consistent Training

The velocity of these particles is parameterized by a neural network. The network is trained to be self-consistent: the velocity it predicts must match the velocity derived from the particle's own carried score. They use a "windowed" approach, breaking the total time into short segments and training the network iteratively.

3. Period-Adapted Sampling & Adaptive Weights

To solve the problem of mass appearing in empty regions, the authors use:

  • Future-looking sampling: They use a cheap, rough approximation of the final result to place particles at $t=0$ in areas where mass will eventually appear.
  • Adaptive Loss: In the training objective, they give extra weight to particles that are predicted to grow significantly in the future, ensuring the network learns the dynamics of emerging modes before they even become large.

Datasets

The method was evaluated on three synthetic but mathematically rigorous benchmark problems:

  • Ornstein–Uhlenbeck (OU) with a linear source: A system with an analytical solution used to validate the accuracy of the carried weights and score.
  • Fisher–KPP Equations (1D and 3D): The classic model for population dynamics and biological invasions, where a population grows until it hits a "carrying capacity."
  • Wasserstein–Fisher–Rao (WFR) Gradient Flows: A complex test case where mass must be moved between distant, initially disconnected regions.

Key Insights and Ideas

  • The "Carried" Score: The most significant "aha" moment is that the score itself satisfies an ODE. By integrating this ODE, the particles maintain a high-fidelity local view of the density field without needing to "talk" to distant particles.
  • Entropy as a Error Bound: The authors prove a relative-entropy inequality. This provides a theoretical guarantee: if you minimize the "self-consistency residual" (how much the network disagrees with the carried score), you are mathematically guaranteed to be bounding the error of the actual density.
  • Importance of "Empty" Space: The paper demonstrates that in non-conservative systems, the initial distribution of particles shouldn't just match the initial density. You must "over-sample" regions that are currently empty but destined for growth.

Key Results

  • High Precision: On the OU and Fisher-KPP benchmarks, the method achieved relative errors for velocity, mass, and score below 0.3% to 1%.
  • Handling New Modes: In the WFR test (the most difficult), the "bottom mode" of the density starts with almost zero mass ($4 \times 10^{-4}$ share) but grows to 20% of the total mass.
  • Ablation Success: Without the period-adapted sampling and adaptive weights, the mass error on the WFR problem was roughly 4%. With these novel mechanisms, it dropped to 0.74%, proving that "future-proofing" the particle placement is essential.

Limitations and Future Work

  • Unbounded Growth: The theoretical error bounds rely on a bounded growth rate. For systems with explosive, unbounded growth (like some OU configurations), the theory doesn't strictly apply, even though the numerical results remain stable.
  • Computational Cost: While it beats the curse of dimensionality compared to grids, calculating high-order spatial derivatives of the neural network for every particle is still computationally expensive.
  • Thin Layers: The method hasn't yet been tested on "singularly perturbed" problems—systems with extremely sharp, moving boundaries (like a shockwave), which might require even more dense particle sampling.
  • Future Direction: The authors suggest exploring adaptive particle management, where particles could be split (when mass grows) or merged (when mass shrinks) on the fly.

For more details, see the full paper: "Score-based neural ordinary differential equations for reaction-diffusions" by Hou and Li (2024).

Summary updated

Abstract

A diffusion equation can be rewritten as a transport equation driven by the score function of the evolving density. Conventional probability-flow formulations, which move particles deterministically with this score-driven velocity, typically conserve total mass. In this work, we develop a Lagrangian method for reaction-diffusion equations in which each particle carries a position and a physical weight. The characteristic ODEs incorporate source terms into the evolution of the log-density and its gradient, while particle weights track changes in mass. We combine this representation with a learned velocity field and weighted particle training to approximate these dynamics. For non-increasing reaction rates, a relative-entropy inequality supports the mass-normalized self-consistency residual as the training objective. This objective can control the error of the induced density when the growth rate is bounded. The algorithm divides the time interval into short windows. In each window, a frozen velocity generates the particle data for a regression of the network velocity. We draw the particles from period-adapted initial sampling distributions and weight them in the loss partly by their predicted future mass, because the reaction can make regions with little initial mass important later. We numerically investigate the method on the Ornstein--Uhlenbeck (OU) equation with a linear source term, Fisher--KPP equations, and Wasserstein--Fisher--Rao gradient flows.
BibTeX
@misc{hou2026score,
  author = {Jianguo Hou and Wuchen Li},
  title = {{Score-based neural ordinary differential equations for reaction-diffusions}},
  year = {2026},
  eprint = {2611.18467},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.18467},
}
Figure