Deep Stochastic Processes via Functional Markov Transition Operators

Jin Xu, Emilien Dupont, Kaspar Märtens, Tom Rainforth, Yee Whye Teh

Abstract

We introduce Markov Neural Processes (MNPs), a new class of Stochastic Processes (SPs) which are constructed by stacking sequences of neural parameterised Markov transition operators in function space. We prove that these Markov transition operators can preserve the exchangeability and consistency of SPs. Therefore, the proposed iterative construction adds substantial flexibility and expressivity to the original framework of Neural Processes (NPs) without compromising consistency or adding restrictions. Our experiments demonstrate clear advantages of MNPs over baseline models on a variety of tasks.

Introduction

Stochastic Processes (SPs), such as Gaussian Processes (GPs), provide a robust framework for modeling distributions over functions and quantifying uncertainty. However, traditional GPs are often limited by their computational complexity—typically scaling cubically with the number of data points—and their reliance on rigid assumptions like Gaussianity. Neural Processes (NPs) were developed to mitigate these issues by combining the flexibility of neural networks with the probabilistic benefits of SPs.

While NPs and their variants (e.g., Attentive NPs, ConvNPs) have shown promise, they often face a fundamental trade-off. Many high-performing "predictive" extensions sacrifice the generative nature of the original formulation, leading to a lack of conditional consistency. This means their predictions may not correspond to a single underlying prior, potentially resulting in miscalibrated uncertainties or conflicting predictions when data is observed sequentially. This paper introduces Markov Neural Processes (MNPs) to bridge this gap, offering a model that is both highly expressive and mathematically consistent.

The Markov Neural Process (MNP) Framework

The core innovation of the MNP is the iterative construction of a stochastic process through functional Markov transitions. Rather than defining a complex SP in a single step, the authors propose stacking neural-parameterized operators that transform a simple, "trivial" SP (such as i.i.d. Gaussian noise) into a complex one.

Functional Markov Transitions

The process is defined as a Markov chain in function space: $f^{(0)} \to f^{(1)} \to \dots \to f^{(T)}$. Each transition $P(f^{(t)} | f^{(t-1)})$ is designed to preserve the essential properties of a valid SP:

  1. Exchangeability: The joint distribution of any finite set of points remains invariant under permutation of the data points.
  2. Consistency: The marginalization of points is well-defined, ensuring that the model's predictions are coherent across different subsets of data.

By proving that these operators maintain these properties, the authors ensure that the final resulting process $f^{(T)}$ is a mathematically valid SP.

Implementation via Normalizing Flows

To achieve high flexibility, the transitions are implemented using Normalizing Flows, specifically Rational Quadratic Splines. This allows the model to warp simple distributions into highly complex, non-Gaussian forms. The inference model operates in reverse, using permutation-invariant architectures like Set Transformers or Deep Sets to estimate latent variables at each step of the chain.

Experimental Evaluation

The authors evaluated MNPs across three distinct domains to test their versatility and robustness.

1D Synthetic Regression

MNPs were tested on standard GP-sampled data and more challenging non-Gaussian datasets, including monotonic functions, convex functions, and samples from Stochastic Differential Equations (SDEs). On convex function data, MNPs achieved a marginal log-likelihood of 5.582, significantly outperforming standard NPs (1.956) and Copula-Based Processes (4.268). This demonstrates the MNP's superior ability to learn and respect functional constraints directly from data.

Contextual Bandits

In the "Wheel" contextual bandit benchmark, MNPs were used to manage the exploration-exploitation trade-off. The model outperformed baselines in both cumulative and simple regret. Notably, as the exploration requirement ($\delta$) increased, the performance gap widened, suggesting that MNPs provide more reliable uncertainty estimates for decision-making in high-stakes environments.

Geostatistics: The GeoFluvial Dataset

The paper introduces the GeoFluvial dataset, containing 25,000 simulations of 2D river patterns. MNPs successfully captured complex spatial structures and produced realistic uncertainty maps, where uncertainty increased in regions distant from measurements. In contrast, standard NPs failed to capture these spatial dependencies, resulting in significantly lower log-likelihoods.

Discussion and Key Insights

The success of MNPs highlights the power of stacking valid transitions. By evolving a simple noise distribution through $T$ steps, the model can represent sophisticated functional forms without sacrificing mathematical rigor.

A critical distinction of the MNP is its "generative" nature. Unlike predictive models that may "forget" the underlying prior as more data is added, MNPs treat context and target points equally. This ensures Bayesian-coherent updates, which is vital for sequential data observation. Furthermore, the use of Normalizing Flows allows the model to handle non-Gaussianity and "kinks" in data that traditional SPs struggle to model.

Conclusion and Future Work

Markov Neural Processes represent a significant step forward in deep generative modeling for stochastic processes. While the current implementation faces some computational overhead due to the stacking of $T$ transitions, future work could explore continuous-time Markov transitions modeled via SDEs to utilize more efficient training methods. Additionally, integrating equivariant convolutions could further enhance performance on spatial and image data by explicitly baking in symmetries like translation invariance.

Summary updated

Abstract

We introduce Markov Neural Processes (MNPs), a new class of Stochastic Processes (SPs) which are constructed by stacking sequences of neural parameterised Markov transition operators in function space. We prove that these Markov transition operators can preserve the exchangeability and consistency of SPs. Therefore, the proposed iterative construction adds substantial flexibility and expressivity to the original framework of Neural Processes (NPs) without compromising consistency or adding restrictions. Our experiments demonstrate clear advantages of MNPs over baseline models on a variety of tasks.
BibTeX
@inproceedings{xu2023deep,
  author = {Jin Xu and Emilien Dupont and Kaspar Märtens and Tom Rainforth and Yee Whye Teh},
  title = {{Deep Stochastic Processes via Functional Markov Transition Operators}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2023},
  url = {https://openreview.net/forum?id=Lg1ODJGGiI},
}
Figure