Abstract
Modeling the evolution of probability densities over time is a fundamental challenge in scientific computing, often addressed using deep learning-based generative models like Normalizing Flows. However, these methods frequently incur high computational costs and lack interpretability. This paper introduces the Dynamic Projection Pursuit Monge Map (D-PPMM), a framework that replaces complex neural networks with numerical linear algebra and Optimal Transport (OT). By mapping snapshots sequentially and using transport splines for interpolation, D-PPMM achieves training speeds orders of magnitude faster than state-of-the-art neural flows while remaining competitive in accuracy across high-dimensional biological and physical datasets.
Introduction
Stochastic processes—systems that evolve over time with inherent randomness—are ubiquitous in fields ranging from cell biology to fluid dynamics. A common task is to build generative models from discrete "snapshots" of these systems to generate new samples or interpolate densities between observed time steps. While Conditional Normalizing Flows (CNFs) are the current standard, they suffer from a "deep learning tax": intensive hyperparameter tuning, non-convex optimization challenges, and high GPU requirements.
Botvinick-Greenhouse et al. propose a more efficient alternative that avoids mapping every snapshot back to a standard Gaussian distribution. Instead, they leverage the intrinsic continuity of physical systems to map snapshots directly to one another using Optimal Transport.
Methodology: The D-PPMM Framework
The core innovation lies in the Dynamic Projection Pursuit Monge Map (D-PPMM), which shifts the generative paradigm from global mapping to sequential transport.
Sequential Mapping vs. Latent Warping
Traditional Normalizing Flows warp a simple latent distribution (like a Gaussian) into a complex target distribution at each time step. In contrast, D-PPMM maps the distribution at time $t_j$ directly to the distribution at $t_{j+1}$. This approach is particularly effective when snapshots are frequent, as the required map is often close to the identity, making it computationally cheap to find.
Projection Pursuit and 1D Optimal Transport
Rather than training a neural network, D-PPMM utilizes the Projection Pursuit Monge Map (PPMM). The algorithm iteratively identifies the most informative 1D directions in high-dimensional space where distributions differ most. It then solves a simple 1D optimal transport problem along these directions. This reliance on numerical linear algebra (such as SVDs and sorting) makes the training process stable, interpretable, and embarrassingly parallelizable.
Temporal Interpolation via Transport Splines
To handle continuous time, the authors employ Transport Splines. By treating the transformation of particles between snapshots as trajectories in Euclidean space, they use cubic splines to interpolate paths. This simplifies the complex task of measure interpolation into a straightforward "connect-the-dots" exercise for particle trajectories, providing clear Lagrangian insights into the system's dynamics.
Experimental Evaluation
The authors evaluated D-PPMM against state-of-the-art Neural Spline Flows (RQ-NSF) across several challenging benchmarks:
- Synthetic Physics: Including the Van der Pol oscillator ($d=2$), Ornstein–Uhlenbeck processes (up to $d=30$), and the chaotic Lorenz-96 system.
- Biological Systems: A large-scale single-cell RNA sequencing dataset tracking Embryoid Body growth over 27 days.
- Animal Behavior: Trajectories of schooling golden shiner fish.
Key Results
- Computational Efficiency: In the Embryoid Body experiment, D-PPMM was 700 times faster to train than RQ-NSF (1.7 seconds vs. 1200 seconds) while achieving comparable accuracy.
- Accuracy in Low Dimensions: For systems with $d < 5$, D-PPMM consistently outperformed neural flows in terms of Maximum Mean Discrepancy (MMD) for equivalent training budgets.
- Scalability: While the performance gap narrows in higher dimensions (up to $d=30$), D-PPMM remains significantly more efficient and easier to tune.
Conclusion and Limitations
D-PPMM offers a mathematically grounded, high-speed alternative to deep generative models for time-series density data. Its primary advantages are stability, interpretability, and extreme computational efficiency. However, the method is subject to certain limitations:
- Error Accumulation: Sequential mapping ($t_1 \to t_2 \to t_3$) can lead to compounding errors over long sequences, a drift issue that latent-mapping flows avoid.
- Data Requirements: The method requires sufficient sample density at each time step and assumes a degree of smoothness ($C^2$ regularity) in the underlying process, making it less suitable for jump processes or highly discontinuous noise.
Despite these constraints, D-PPMM represents a significant step toward making high-dimensional generative modeling accessible without the overhead of deep learning optimization.
Summary updated