TL;DR
This paper introduces Deep Random Splines (DRS), a novel deep generative model for random functions that can enforce shape constraints like nonnegativity. They apply DRS to model the intensity functions of neural population spike trains, achieving better dimensionality reduction and representation of neural activity compared to existing methods.
What Problem Does This Paper Address?
Modeling random functions is crucial in many scientific domains, including neuroscience where we often want to understand the underlying processes that generate observed data, like neural spikes. Gaussian Processes (GPs) are a popular tool for this, but they struggle with computational scaling for large datasets and enforcing specific shape constraints (e.g., ensuring an intensity function is always positive). Splines, on the other hand, are good at enforcing shape constraints but are less flexible for complex probabilistic modeling. This paper bridges this gap by proposing a method that combines the strengths of deep generative models with the constraint-enforcing capabilities of splines.
Methods
The core innovation is Deep Random Splines (DRS). DRS models a random function by transforming standard Gaussian noise through a deep neural network. The output of this network is not the function itself, but rather the parameters of a spline. This allows the model to generate complex, flexible random functions while also enabling the enforcement of desired shape constraints on the spline.
To enforce nonnegativity on the spline's intensity function, the authors leverage a specific parameterization of nonnegative splines based on symmetric positive semidefinite matrices. They then employ the Method of Alternating Projections to ensure that the neural network's output adheres to these constraints. This iterative projection method allows them to approximately compute the necessary projections, making it differentiable for training.
For inference on neural population data, they propose a variational autoencoder (VAE) architecture. A key novelty here is a recurrent encoder that can handle multiple point processes (representing different neurons) as input, allowing for a shared low-dimensional representation of each trial.
Datasets
The paper uses both simulated data and two real-world neural recording datasets:
- Simulated Data: Generated to evaluate the model's ability to recover true intensity functions and low-dimensional representations of trials.
- Reaching Data: A dataset collected by Churchland et al. [8] consisting of neural recordings from a primate performing a reaching task. This dataset involves 20 neurons recorded over 3590 trials.
- Cycling Data: A newly collected dataset where a primate navigates a virtual environment using a hand-held pedal. This dataset includes recordings from 20 neurons across 1300 trials, with 8 distinct trial types.
Key Insights and Ideas
- Bridging the Gap: DRS successfully combines the flexibility of deep generative models with the constraint-enforcing power of splines, addressing limitations of both GPs and traditional spline methods.
- Shape Constraints are Key: Enforcing nonnegativity on neural intensity functions is crucial for interpretability and model stability, and DRS provides a principled way to do this.
- Low-Dimensional Representations: The VAE framework with the novel recurrent encoder effectively captures trial-specific dynamics in a low-dimensional latent space, revealing distinct clusters for different trial types.
- Splines for Intensity: Splines are a natural choice for modeling point process intensities due to their analytical integrability, which simplifies likelihood calculations compared to GP-based methods.
Key Results
- Simulated Data: DRS-VAE achieved a higher per-trial ELBO (Evidence Lower Bound) than competing methods like PP-GPFA, PfLDS, and GPFA. It also demonstrated superior recovery of true intensity functions, as measured by L2 distance, with statistically significant improvements. The model successfully clustered trials based on their type, indicating effective dimensionality reduction.
- Reaching Data: DRS-VAE again showed a higher ELBO. While PfLDS had more than 20x the latent dimensions, DRS-VAE still outperformed it in predicting trial types using 15-nearest neighbors on posterior means and in explaining variation via ANOVA. The posterior intensity samples from DRS-VAE were observed to be smoother and more plausible.
- Cycling Data: While the ELBO was slightly lower than PfLDS (attributed to data preprocessing artifacts), DRS-VAE still outperformed PP-GPFA and showed significantly better latent representation quality for trial type prediction and explained variation, despite having much lower-dimensional latent spaces.
Limitations and Future Work
- Computational Cost for Spikes: The LSTM-based encoder, while effective, has a computational cost that scales with the number of spikes. The authors suggest non-amortized inference as a potential solution for very large datasets.
- Generalizing to Multi-dimensional Point Processes: The current framework is for univariate point processes. Extending it to multi-dimensional point processes (e.g., spatial point processes) is a promising future direction, potentially by replacing splines with other easily-integrable nonnegative functions like conic combinations of Gaussian kernels.
- More Complex Point Processes: The model could be extended to more complex point processes like Hawkes processes, where the intensity in an interval depends on previous spiking history.
- Broader Applicability: DRS can be applied to any setting requiring random functions, not just point processes, opening up many potential applications beyond neural data analysis.
- Other Constraints: While nonnegativity and smoothness were enforced, the framework can be extended to enforce other constraints like monotonicity, convexity, or concavity.
Summary updated