Rescaling, thinning or complementing? On goodness-of-fit procedures for\n point process models and Generalized Linear Models

Felipe Gerhard, Wulfram Gerstner

Abstract

Generalized Linear Models (GLMs) are an increasingly popular framework for modeling neural spike trains. They have been linked to the theory of stochastic point processes and researchers have used this relation to assess goodness-of-fit using methods from point-process theory, e.g. the time-rescaling theorem. However, high neural firing rates or coarse discretization lead to a breakdown of the assumptions necessary for this connection. Here, we show how goodness-of-fit tests from point-process theory can still be applied to GLMs by constructing equivalent surrogate point processes out of time-series observations. Furthermore, two additional tests based on thinning and complementing point processes are introduced. They augment the instruments available for checking model adequacy of point processes as well as discretized models.

Introduction

In computational neuroscience, modeling the precise timing of action potentials (spikes) is crucial for understanding neural coding. While neural activity is fundamentally a continuous-time point process, experimental data is frequently recorded and analyzed in discrete time bins using Generalized Linear Models (GLMs). A significant challenge arises when validating these models: the standard "gold standard" for goodness-of-fit, the Time-Rescaling Theorem, assumes continuous time. When applied to binned data or high-frequency firing, this assumption fails, leading to systematic biases that can cause researchers to incorrectly reject valid models.

Methodology

The researchers propose a framework to bridge the gap between discrete-time GLMs and continuous-time point process theory. This involves two main components: the generation of surrogate point processes and the introduction of novel statistical tests.

Surrogate Point Processes

To address discretization bias, the authors developed a method to transform discrete GLM outputs into continuous-time surrogate spike trains. For Poisson GLMs, spikes are randomly distributed within each bin. For Bernoulli GLMs, a biased Poisson distribution is used to estimate the underlying number of spikes before distributing them. This mapping allows the application of continuous-time goodness-of-fit procedures to discrete data.

Goodness-of-Fit Procedures

The paper evaluates three primary methods for assessing model adequacy:

  1. Time-Rescaling: The traditional approach that transforms the time axis based on the model's predicted intensity to achieve a homogeneous Poisson process.
  2. Random Thinning: A "local" test that deletes observed spikes with a probability proportional to the predicted firing rate. If the model is correct, the remaining spikes form a constant-rate Poisson process.
  3. Random Complementing: The inverse of thinning, where simulated spikes are added in regions where the model predicts low firing rates. This test is particularly sensitive to the shape of the intensity function between actual spikes.

Results

The proposed methods were benchmarked using synthetic datasets, including Inhomogeneous Poisson processes, Renewal processes, and complex Spike Response Models (SRM).

Key findings include:

  • Sensitivity Boost: Thinning and complementing tests demonstrated significantly higher sensitivity to model errors than the standard time-rescaling theorem. In several simulations, these new tests detected deviations twice as small as those required by the traditional method.
  • Complementing as a Powerful Tool: The complementing test emerged as a particularly robust method, showing high sensitivity to the exact shape of the predicted intensity function, even during quiet periods between spikes.
  • Bias Mitigation: The surrogate point process method effectively eliminated the discretization bias, preventing the false positives that typically plague binned data analysis.
  • ROC Analysis: Receiver Operating Characteristic (ROC) curves confirmed that thinning and complementing provide superior detection power for a given false-alarm rate compared to time-rescaling.

Discussion

While the new methods offer substantial improvements, they come with certain considerations. Both thinning and complementing require the selection of firing rate thresholds, which the authors manage using Simes' procedure for multiple testing. Additionally, the surrogate method assumes a constant firing rate within a single bin, an assumption that may weaken if bins are excessively large.

Future work will involve applying these robust "stress tests" to experimental neural recordings to evaluate the accuracy of existing GLMs in the literature. By providing more sensitive and bias-free tools, this research enhances the ability of neuroscientists to validate their mathematical models of neural activity.

Summary updated

Abstract

Generalized Linear Models (GLMs) are an increasingly popular framework for\nmodeling neural spike trains. They have been linked to the theory of stochastic\npoint processes and researchers have used this relation to assess\ngoodness-of-fit using methods from point-process theory, e.g. the\ntime-rescaling theorem. However, high neural firing rates or coarse\ndiscretization lead to a breakdown of the assumptions necessary for this\nconnection. Here, we show how goodness-of-fit tests from point-process theory\ncan still be applied to GLMs by constructing equivalent surrogate point\nprocesses out of time-series observations. Furthermore, two additional tests\nbased on thinning and complementing point processes are introduced. They\naugment the instruments available for checking model adequacy of point\nprocesses as well as discretized models.\n
BibTeX
@inproceedings{gerhard2010rescaling,
  author = {Felipe Gerhard and Wulfram Gerstner},
  title = {{Rescaling, thinning or complementing? On goodness-of-fit procedures for\textbackslash\{\}n point process models and Generalized Linear Models}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2010},
  url = {https://openreview.net/forum?id=Bk-6t8W_bB},
}

Loading related content...

Figure