Joint Posteriors over Latent Dynamics and Spike-History Filters by Expectation Propagation

When a neuron fires, its own spike changes what it does next. It falls briefly silent (refractoriness), fires in bursts, or slowly adapts. Neuroscientists also model population recordings with a slowly varying hidden "latent state," such as an evolving decision variable. The trouble is that both explanations can account for the same dip or surge in firing. A post-spike silence can be read as the neuron's own refractoriness or as a drop in the hidden state. Zoltowski et al. showed the stakes on macaque area LIP (lateral intraparietal cortex): leave out spike history and continuous ramping trajectories get misread as discrete steps. Current inference tools do not report how uncertain this split is. Variational methods such as vLGP fit a Gaussian posterior that is known to be overconfident, and they typically keep the history filter separate from the latent path. Exact sampling, as in Zoltowski et al., is accurate but does not scale beyond one-dimensional latents.

This note proposes using expectation propagation (EP) to approximate a single joint Gaussian posterior over the latent trajectory and the history filter. EP is a deterministic approximate-inference method that repeatedly replaces each hard likelihood term with a Gaussian chosen to match means and variances. Its output answers two scientific questions directly. The first is how strongly the data tie a latent "step" to a refractory dip. The second is which latent-dynamics hypothesis predicts held-out spikes better. EP's "cavity" distributions provide a nearly free leave-one-out estimate for that second comparison.

Core Claim

The structure that makes this tractable is easy to miss. In a discrete-time point-process model, the log firing rate of neuron $n$ in bin $t$ is

$$\eta_{nt} = c_n^\top x_t + h_n^\top y^{\text{hist}}_{nt} + b_n .$$

Here $x_t$ is the latent state, $c_n$ is the neuron's loading onto it, $h_n$ is its post-spike history filter (expressed in a small basis of raised-cosine functions), $y^{\text{hist}}_{nt}$ is the vector of that neuron's own observed past spikes projected onto the basis, and $b_n$ is a baseline. Past spikes are observed, so for fixed loadings $c_n$, $\eta_{nt}$ is a linear function of the unknowns $(x_t, h_n, b_n)$. Every per-bin likelihood factor therefore depends on the joint posterior through one scalar.

As a result, EP's moment-matching step is an exact one-dimensional quadrature in every bin. This is the property Gerwinn et al. exploited for static GLMs, and it holds here even though the unknowns now include a whole latent path. The joint Gaussian has an "arrowhead" structure: the latent path has a banded (Markov) block, the history and baseline weights form a dense block of size $P = N\times L$, and only the spike factors connect the two. A Schur-complement solve handles this in roughly $O(Td^3 + Td^2P + P^3)$ per sweep, where $T$ is the number of time bins, $d$ is the latent dimension, $N$ is the number of neurons, and $L$ is the number of basis functions. That is cheap enough for tens of neurons over hundreds of trials.

The claim is twofold. First, joint EP recovers the posterior coupling between the history filter and the latent path, measured against a sampling reference. Factorized variational inference misses this coupling, and the gap widens exactly where the two explanations compete: slow adaptation versus slow drift, and sparse spiking. Second, EP's cavity-based leave-one-out scores (Vehtari et al.) are accurate enough to compare stepping and ramping dynamics at population scale, a comparison that today requires sampling.

Relation to Prior Work

Two papers supply the method. Gerwinn et al. ran EP on GLMs with history and coupling filters under a Laplace prior and obtained ±3σ intervals per weight. Their weights were static, with no latent dynamics. We add a latent Markov path and study the cross-covariance it creates. Cseke, Opper and Sanguinetti derived EP for latent Gauss–Markov processes observed through point processes and showed that their skewness corrections match sampling on rat taste cells. Their observation model is a Cox process with no history kernel. We keep discrete time to avoid the near-singular hazard their continuous limit would face under hard refractoriness, which is the tension the file flags.

Two other papers motivate the gap. vLGP already combines history-dependent observations with a latent Gaussian-process prior, using reverse-KL variational inference. The question is not whether such models can be fit but whether their uncertainty is right. Zoltowski et al. show that the history/latent split changes scientific conclusions. Their sampling-based WAIC and leave-one-out analysis is our gold standard, and it is limited to one-dimensional latents.

One paper constrains the design. Deisenroth and Mohamed found iterative EP over dynamical states to be about two orders of magnitude slower than linearized smoothing, at roughly 20 s versus under 0.5 s per time slice. That gap arose from moment matching through nonlinear transitions. Our transitions are linear-Gaussian and the only non-Gaussian terms are one-dimensional, so we expect, and will measure, a far smaller gap. EP's lack of general convergence guarantees is the other constraint. Our sites are log-concave, so we use damped, parallel updates with precision clipping and report non-convergence rates.

Research Program

We will build the arrowhead-structured EP solver for latent linear dynamical systems and latent Gaussian processes with per-neuron history filters, using a Poisson likelihood with exponential or softplus nonlinearity. Loadings and dynamics hyperparameters are learned by maximizing EP's evidence approximation. We then add the post-hoc skewness correction of Cseke et al. for marginals of interest, along with cavity-based leave-one-out predictive scores.

The analysis has three parts. The first is a synthetic study with known ground truth, in which a slow history kernel (adaptation) and slow latent drift are made deliberately confusable. The second is a check that approximate leave-one-out scores pick the same model as sampling. The third reanalyzes stepping versus ramping with multi-dimensional latents and spike history jointly uncertain.

Baselines

The naive reference point is a latent model with no history filter, fit by EP. It quantifies how much of the latent structure is history artifact. The static-GLM end of the spectrum is EP without latents, following Gerwinn et al.. On rabbit retinal ganglion cells that paper reports a mean predicted-versus-actual rate correlation of 0.3699, against 0.1924 for the spike-triggered average. If we can obtain that data, our latent-augmented model must exceed 0.3699 under the same splits to show that latents add anything there.

The strongest existing scalable method is vLGP. It reports a pseudo-R² of 20.88% on 63 V1 neurons, against 6.77% for a per-neuron-noise baseline. We will run vLGP and our EP on identical trials, latent dimension, basis functions, and hyperparameter search budget, with equal wall-clock limits. Beating it means two things: matching or exceeding its held-out predictive likelihood, and getting substantially closer to the sampling reference on credible-interval coverage and on the latent–history cross-covariance. Matching prediction alone is not enough, since the claim concerns calibration.

A Laplace approximation on the same arrowhead structure is the closest competitor. Newton's method around the MAP estimate yields a joint Hessian that also contains cross-terms. It is run with identical priors.

Long-run sampling (Hamiltonian Monte Carlo, or the samplers used by Zoltowski et al.) is the accuracy reference, not a competitor. We report the cost ratio against it.

The key ablation is EP with the history-to-latent cross-covariance forced to zero, implemented as a block-diagonal projection. It isolates whether the joint coupling, rather than moment matching in itself, drives any calibration gain.

Why not just…?

Why not just use MAP estimation with L1/L2 penalties? Point-process GLMs with history have log-concave likelihoods (Paninski et al.), so MAP is fast and globally optimal. It returns no uncertainty, though, and the question here is precisely how uncertain the history/latent split is. Penalized MAP serves as our predictive-accuracy floor. Separately, Park and Pillow report that their Bayesian spike-triggered covariance method needed two orders of magnitude less data than maximum likelihood. That is an argument for Bayesian regularization in scarce-data regimes, not for EP specifically.

Why not just use variational inference as in vLGP? It scales and avoids site-division instabilities. Mean-field VI cannot represent the cross-covariance we target. Structured VI could, but the reverse-KL objective still shrinks variance. The decisive experiment is the synthetic confound study: if structured full-covariance VI matches EP's coverage against sampling, EP's advantage reduces to the leave-one-out byproduct, and we will say so.

Why not just use a Laplace approximation on the joint posterior? This is the real threat. Laplace captures cross-terms at the same cost, and for log-concave Poisson models the posterior may be close to Gaussian. We expect EP to win only in low-count bins and near hard refractoriness, where the per-bin factor is skewed, and in leave-one-out estimation, where Laplace has no cavity. We will measure where in firing-rate space the two diverge. If they never diverge meaningfully, the contribution becomes "joint Laplace suffices, and here is the calibration evidence," which is still a useful corrective to factorized VI.

Why not just train a spiking network with autaptic feedback using surrogate gradients? Models such as STC-LIF cut Moving MNIST MSE from 102.8 to 47.0, but they are predictive point estimators. They offer no posterior or model-comparison criterion for deciding between stepping and ramping, so they answer a different question.

Proposed Experiments

The synthetic study comes first. We simulate populations of 5–40 neurons, each with a ramping or stepping one- to three-dimensional latent and a history kernel whose timescale overlaps the latent's autocorrelation, at mean rates from 2 to 50 Hz. Against long-run sampling we measure four things: KL divergence of the latent and history marginals, 90% interval coverage of both the true latent path and the true history weights, error in the estimated latent–history posterior correlation, and wall-clock time. The decisive check is whether, in the most confounded settings, EP's posterior correlation tracks sampling while factorized VI and the block-diagonal ablation report confident but wrong separations.

Second, we test model comparison. In the same simulations we check whether EP cavity leave-one-out scores select the generating dynamics (stepping or ramping) as often as sampling-based PSIS-LOO and WAIC, and whether Laplace and VI evidence do.

Third, we turn to real data. On the 63-neuron V1 drifting-grating recordings used by vLGP, we compare held-out pseudo-R² against the reported 20.88% and check calibration via held-out spike predictive coverage. If the LIP recordings of Zoltowski et al. can be obtained, we refit their variable-duration task with multi-dimensional latents and jointly uncertain history. Their reported reference is that with spike history added to linear models, 28 of 40 neurons favored stepping, while a ramping model with baseline and compressive nonlinearity accounted for roughly half. We ask how many of those classifications remain decisive once history uncertainty is propagated. Any classification that EP's leave-one-out cannot separate from ramping would be a concrete revision of the scientific claim.

For runtime, we report EP-to-Laplace and EP-to-VI time ratios. A ratio well under the ~40× GPDS gap would confirm that the one-dimensional-site structure removes EP's usual cost penalty.