Gaussian Process Priors for Latent Trajectories

You are reading version 1 of this direction, frozen 2026-10-04. The latest version is version 2.

Version 1 of 2121 papers146 conceptsfrozen 2026-10-04

In many real-world datasets—from the rhythmic beating of a heart to the evolving themes in a century of literature—the most important information is hidden. We use latent variable models like Variational Autoencoders (VAEs) or Factor Analysis to uncover these "hidden" signals. However, standard versions of these models often treat each observation as an independent snapshot, ignoring the fact that the world usually moves in smooth, continuous paths.

To bridge this gap, researchers have increasingly turned to Gaussian Process (GP) priors. By placing a GP directly on the latent space, we can enforce temporal consistency, handle irregularly sampled data, and ensure that our models respect the physical or biological "slowness" of the systems they describe.

The Problem of Independent Latents

When we map high-dimensional data (like video frames or neural spikes) into a low-dimensional latent space, we want that space to be organized. In a standard VAE, the prior is typically a simple Gaussian distribution, $p(z) = \mathcal{N}(0, I)$. This tells the model that any latent point is as likely as any other, regardless of what happened a millisecond ago.

As noted by Liang Xu and colleagues in ActFormer, this lack of temporal structure often leads to "drift." In human motion generation, a model might start a sequence realistically but eventually collapse into a static, average pose because it has no mathematical "memory" of the trajectory it should be following.

Gaussian Processes as Temporal Glue

A Gaussian Process is not just a distribution over numbers, but a distribution over functions. By using a GP as a prior for latent variables, we define how those variables should correlate over time. This is controlled by a kernel function, which acts as a dial for the "smoothness" of the latent path.

  1. Enforcing Smoothness: In Variational Inference for Uncertainty on the Inputs of Gaussian Process Models, Andreas C. Damianou and Neil D. Lawrence demonstrated that placing a temporal GP prior over the latent space constrains variables to follow continuous dynamics. This is essential for capturing the underlying structure of sequential observations.
  2. Flexible Dynamics: Unlike simple random walks, GPs can capture diverse patterns. Florian Wenzel and Stephan Mandt showed in Scalable Generalized Dynamic Topic Models that by selecting specific kernels, models can represent everything from smooth transitions to long-term dependencies or sudden, "bursty" events.

Unifying Dimensionality Reduction and Smoothing

Traditionally, researchers handled noisy time-series data in two steps: first, reduce the dimensions, and then apply a filter (like a Kalman filter) to smooth the result. Byron M. Yu and Maneesh Sahani argued in Gaussian-process factor analysis for low-dimensional single-trial analysis of neural population activity that this is suboptimal.

Their framework, GPFA, performs both steps simultaneously. By mapping high-dimensional neural spikes to a latent space governed by GP priors, the model learns the optimal smoothing parameters and the spatial mapping at the same time. This principled approach allows for the estimation of continuous "firing rates" from discrete, noisy events, as further explored by John Patrick Cunningham in Inferring Neural Firing Rates from Spike Trains Using Gaussian Processes.

Handling Irregularity and Continuity

One of the most powerful features of the GP prior is its ability to handle "missing" data. Because a GP defines a continuous function, it doesn't care if your observations are spaced one second or one hour apart.

In A Review of Dynamic Network Models with Latent Variables, Bomin Kim and colleagues highlight how GPs allow latent node trajectories in social networks to be modeled as continuous functions. This moves beyond the limitations of discrete-time Markov chains, providing a principled way to model temporal dependencies across arbitrary intervals. Similarly, in Multi-modal Gaussian Process Variational Autoencoders, researchers found that this continuity makes models robust to occlusions or missing frames in video data.

Beyond Smoothing: Generative Streams

The utility of GP priors has recently expanded into generative modeling, specifically Flow Matching and Diffusion. Standard diffusion models often start with independent Gaussian noise at every pixel or time step. However, for time-series data, this ignores the inherent correlation between adjacent points.

In Stream-level Flow Matching with Gaussian Processes, Ganchao Wei and Li Ma replace linear paths with stochastic "streams" modeled by GPs. By leveraging the analytical properties of GPs, they achieve smoother vector fields and reduced sampling variance. This ensures that the generated data doesn't just look like the target distribution at the end, but follows a physically plausible trajectory throughout the entire generation process.

Physical and Biological Realism

Finally, GP priors act as a form of "soft" physical constraint. In Bidirectional Recurrence for Cardiac Motion Tracking with Gaussian Process Latent Coding, Jiewen Yang and colleagues use GPs to ensure that tracked heart tissue follows mathematically smooth and physically consistent paths. By constraining the latent representation, the model prevents unrealistic deformations that a purely data-driven approach might suggest.

Whether it is identifying transcription factors in gene regulatory networks or reconstructing the paths of cells in Smooth Schrödinger Bridges, the core intuition remains the same: the latent world is continuous, and the Gaussian Process is the most effective tool we have to describe that continuity.

Papers in this version

Showing 60 of 121 papers.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 92 papers
  2. Version 1 · 2026-10-04 · 121 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_temporal_smoothness_priors_2c29da3d,
  title = {temporal smoothness priors},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/temporal-smoothness-priors/rdr_2c29da3db36c47388d5501e10547c6a5},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_gaussian_process_priors_for_latent_traje_2c29da3d_v1,
  title = {Gaussian Process Priors for Latent Trajectories},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/gaussian-process-priors-for-latent-trajectories/rdr_2c29da3db36c47388d5501e10547c6a5/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}