Deep Random Splines for Point Process Intensity Estimation

Gabriel Loaiza-Ganem, John P. Cunningham

Introduction

Modeling the rhythm of neural spikes and other point processes is a central challenge in computational neuroscience. The authors introduce Deep Random Splines (DRS), a novel framework that combines the flexibility of deep generative models with the structural guarantees of mathematical splines. By transforming Gaussian noise through a neural network into spline parameters, DRS efficiently estimates non-negative, smooth intensity functions for point processes (like neural firing rates), outperforming traditional Gaussian Process-based methods.

Background & Motivation

In fields like neuroscience, researchers frequently need to model "random functions"—such as the fluctuating firing rate (intensity) of a neuron across different trials of an experiment.

The standard tool for this task is the Gaussian Process (GP). However, GPs suffer from two major limitations:

  1. Scalability: They scale poorly with the number of data points, exhibiting $O(N^3)$ computational complexity.
  2. Shape Constraints: It is mathematically difficult to force a GP to be strictly non-negative or monotonic without making the underlying math incredibly complex or the inference prohibitively slow.

While splines (piecewise polynomials) are excellent for enforcing these specific shapes, they are traditionally treated as fixed curves rather than distributions. This paper bridges that gap, proposing a rich, learnable distribution over functions that are guaranteed to be both smooth and non-negative.

Deep Random Splines (DRS) Methodology

The core idea behind DRS is a generative pipeline that ensures mathematical constraints are met by design:

  1. Latent Mapping: The process begins with a simple latent variable $Z$ (Gaussian noise).
  2. Neural Transformation: $Z$ is passed through a deep neural network to produce the parameters $\psi$ of a spline.
  3. Constraint Satisfaction: To ensure the resulting spline is non-negative and smooth (having continuous derivatives at the "knots"), the authors use a clever parametrization of polynomials. Because satisfying multiple constraints simultaneously is difficult, they employ the Method of Alternating Projections. This iteratively "nudges" the parameters until they lie in the intersection of all required convex sets (e.g., non-negativity and continuity).
  4. Inference via VAE: This generative process is wrapped in a Variational Autoencoder (DRS-VAE). To handle inputs consisting of continuous-time event data rather than fixed grids, the authors designed a recurrent encoder using LSTMs for each point process.
Recurrent encoder architecture using LSTMs for each point process.

A key advantage of using splines is that, as polynomials, their integrals can be computed analytically (in closed form). This is a massive computational win for calculating the log-likelihood of a Poisson process, which requires integrating the intensity function over time.

Experimental Setup

The model was evaluated on three distinct types of data:

  • Simulated Data: Synthetic point processes generated from exponentiated Gaussian Processes to provide a ground truth for intensity recovery.
  • Reaching Data: Neural recordings from a primate performing reaching movements to 40 different locations.
  • Cycling Data: A dataset involving a primate pedaling at various speeds and directions.

For the real-world datasets, a "group lasso" preprocessing step was used to select the 20 most informative neurons, keeping training times manageable.

Key Insights

  • Differentiable Projections: By using the Method of Alternating Projections, the authors can differentiate through the constraint-satisfaction process. This allows the neural network to learn how to generate "almost correct" parameters that only need a few projections to become valid splines.
  • Continuous Time vs. Binning: Most competing models (like PfLDS) discretize time into "bins" (e.g., 20ms windows). DRS works with exact, continuous event times, avoiding the information loss inherent in binning.
  • Implicit Smoothness: Unlike GPs, which require a carefully chosen kernel to enforce smoothness, DRS enforces smoothness through the spline architecture itself. This allows the latent space to focus entirely on capturing the underlying trial dynamics.

Results & Performance

The DRS-VAE was compared against GPFA (Gaussian Process Factor Analysis) and PfLDS (Poisson-feedforward Linear Dynamical System).

  • Higher Fidelity: On simulated data, DRS recovered the true intensity functions with significantly lower $L^2$ error than its competitors.
  • Better Representations: On the "Reaching" task, DRS achieved 23.73% accuracy in predicting the reach target from just a 2D latent space. PfLDS, despite having access to more dimensions, only achieved 3.05%.
  • Efficiency: In the "Cycling" task, DRS outperformed PfLDS in trial-type classification (55.85% vs 11.70%) while using a latent representation that was 28 times smaller.
  • Visual Quality: Qualitatively, the intensity functions produced by DRS are much smoother and more biologically plausible than the "boxy" intensities produced by binned methods.
Comparison of DRS-VAE (smooth) and PfLDS (boxy) posterior samples against true intensity.

Discussion & Future Work

While DRS presents a significant step forward, it has limitations. The current work focuses on 1D functions (time); extending this to multi-dimensional splines (e.g., for spatial point processes) would be computationally demanding. Additionally, the architecture requires one LSTM per neuron. While effective for 20-50 neurons, scaling to modern probes with thousands of neurons will require a more unified encoder architecture.

Finally, the authors suggest that future work could replace the underlying Poisson process with more complex models, such as Hawkes processes, where the occurrence of one event dynamically changes the probability of future events.

Summary updated

BibTeX
@inproceedings{loaizaganem2019deep,
  author = {Gabriel Loaiza-Ganem and John P. Cunningham},
  title = {{Deep Random Splines for Point Process Intensity Estimation}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2019},
}
Figure