Markovian Gaussian Process: A Universal State-Space Representation for Stationary Temporal Gaussian Process

Weihan Li, Yule Wang, Chengrui Li, Anqi WuarXiv 2024 · DOI · arXiv

Introduction

Understanding the dynamic interactions between brain regions is a central challenge in systems neuroscience. Communication between these regions is not instantaneous; signals propagate with delays that shift according to cognitive state, attention, and stimulus properties. However, existing models often assume static delays or fail to scale to the massive datasets generated by modern recording technologies like Neuropixels. Gaussian Processes (GPs), while expressive, typically suffer from $O(T^3)$ computational complexity, making them impractical for long-duration recordings.

The Adaptive Delay Model (ADM)

To address these challenges, researchers at Georgia Tech developed the Adaptive Delay Model (ADM). This framework introduces three key innovations:

  1. The GP-SSM Bridge: A theoretical proof that any temporally stationary GP can be represented as a Multi-Order State Space Model (SSM). This allows the model to retain the expressivity of GPs while leveraging the computational efficiency of SSMs.
  2. Time-Varying MOSE Kernels: By using a Multi-output Squared Exponential (MOSE) kernel with a time-varying delay parameter, ADM can continuously track shifts in communication latency.
  3. Parallel Scan Kalman EM: By replacing sequential processing with a parallel scan algorithm, the authors reduced temporal complexity from $O(T)$ to $O(\log T)$, enabling efficient use of modern GPU hardware.

Synthetic Validation and Scaling

The model's ability to recover ground-truth dynamics was first tested on synthetic data. ADM successfully tracked shifting delays (e.g., transitions between 1 and 5 time bins) with high precision, achieving Pearson correlation coefficients of approximately 0.9. Crucially, the parallelized inference demonstrated significant speedups over sequential methods as the number of time bins increased, proving its utility for "big neural data."

Figure 1: Synthetic data validation showing (A) accurate recovery of time-varying delays and latent signals, (B) test log-likelihood across model orders, and (C) the logarithmic scaling advantage of parallelized inference.

Real-World Neural Dynamics

ADM was applied to simultaneous V1-V2 recordings from macaques and five-region Neuropixels data from mice. In the V1-V2 dataset, ADM identified a strong feedback signal from V2 to V1 immediately following stimulus onset, likely representing a "prediction error" signal that stabilizes as the stimulus becomes predictable. In the mouse data, the model's inferred communication directions perfectly aligned with known anatomical hierarchies, identifying VISp (primary visual cortex) as the hierarchical source and VISam as the endpoint.

Figure 6: Performance on monkey V1-V2 data. (A) ADM tracks dynamic delays and communication directions. (B-D) ADM outperforms baselines like DLAG and MRM-GP in both predictive accuracy (log-likelihood) and computational efficiency.

Conclusion and Future Directions

ADM provides a scalable, expressive framework for mapping the brain's changing conversations. While it solves the temporal scaling problem ($T$), future work is needed to address the $O(N^3)$ spatial complexity associated with increasing the number of brain regions ($N$). Potential solutions include frequency-domain techniques to achieve linear scaling in latent size and the use of non-stationary kernels to capture even more complex neural dynamics. This work paves the way for real-time analysis of large-scale neural circuits and advanced brain-computer interfaces.

Summary updated

BibTeX
@misc{li2024markovian,
  author = {Weihan Li and Yule Wang and Chengrui Li and Anqi Wu},
  title = {{Markovian Gaussian Process: A Universal State-Space Representation for Stationary Temporal Gaussian Process}},
  year = {2024},
  eprint = {2407.00397},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2407.00397},
}
Figure