Rescaling, thinning or complementing? On goodness-of-fit procedures for point process models and Generalized Linear Models

Felipe Gerhard, Wulfram Gerstner

Evaluating the accuracy of neural spike train models is a fundamental challenge in computational neuroscience. How do we know if our model of a neuron’s firing pattern is actually any good? This paper addresses this issue by providing a robust framework for applying continuous-time statistical tests to discrete-time neural models, while also introducing two novel goodness-of-fit tests that often outperform industry standards.

The Discretization Problem in Neural Modeling

In neuroscience, neurons are frequently modeled as point processes, where the primary variable of interest is the exact time a spike occurs. The gold standard for validating these models is the time-rescaling theorem. This theorem mathematically transforms a complex, time-varying spike train into a simple, predictable "unit-rate Poisson process"—provided the underlying model is correct.

However, modern researchers often rely on Generalized Linear Models (GLMs), which operate on binned, discrete data (e.g., counting how many spikes occurred in a specific 1ms window). Applying continuous-time theorems to this discrete data introduces "discretization bias." If the firing rate is high or the bins are too large, standard tests break down, potentially validating a model that actually has significant flaws.

Bridging the Gap: Surrogate Point Processes

To resolve the incompatibility between discrete GLMs and continuous-time tests, the authors propose a method to "re-constitute" continuous spike times from binned data by generating surrogate point processes.

  • For Poisson-GLMs: The observed number of spikes is distributed uniformly within each bin.
  • For Bernoulli-GLMs (binary 0/1 data): A biased Poisson distribution is used to estimate the actual number of spikes that occurred in that bin before distributing them.
Flowchart for generating surrogate point processes from discrete GLMs.

By generating these surrogates, researchers can effectively remove the systematic bias caused by simply plugging binned GLM probabilities into the time-rescaling theorem.

Two New Statistical Tests

Beyond fixing the discretization issue, the paper introduces two novel goodness-of-fit tests that augment the traditional time-rescaling approach:

  1. Random Thinning: This method involves probabilistically "pruning" the observed spikes. A spike is kept with a probability inversely proportional to the model's predicted intensity. If the model is accurate, the remaining spikes will resemble a perfectly steady, low-rate Poisson process.
  2. Random Complementing: This is the inverse approach. It "fills in the gaps" by adding simulated spikes to the real ones until the total rate reaches a constant upper bound. Again, an accurate model will result in a combined spike train that forms a steady Poisson process.
Visual comparison of time-rescaling, random thinning, and complementing methods.

Key Insights and Performance

Using synthetic datasets with known "ground truths" (including Inhomogeneous Poisson Processes, Renewal Processes, and Inhomogeneous Spike Response Models), the authors measured the "test power" of these methods—their ability to catch a flawed model.

  • Complementary Strengths: The time-rescaling theorem excels at checking if the model captured the distribution of intervals (the neuron's "rhythm"). Conversely, thinning and complementing are superior at detecting errors in modulation (how the neuron responds to external stimuli). Thinning focuses on the exact moments of firing, while complementing evaluates the entire shape of the predicted intensity curve, including quiet periods.
  • Higher Sensitivity: For the Inhomogeneous Poisson and Spike Response models, thinning and complementing detected model errors that were half the size of those required to trigger a failure in the standard time-rescaling test.
  • ROC Analysis: Receiver Operating Characteristic (ROC) curves demonstrated that the new methods provide a better trade-off between false alarms and successful detections across various levels of model error. (The time-rescaling theorem remained slightly superior only for the Renewal Process, confirming its specialized strength for inter-spike interval distributions).

Limitations and Future Directions

One limitation of thinning and complementing is the requirement to set lower and upper bounds for the firing rate. If these bounds are too loose—such as when a neuron's rate drops near zero—the tests lose statistical power because they evaluate too few spikes. To mitigate this, the authors introduced a "stitching" procedure that concatenates intervals where the rate is sufficiently high.

Future work may involve automating the refinement of these thresholds or extending these goodness-of-fit procedures to population models, where multiple neurons fire and interact simultaneously.

Summary updated

Abstract

Generalized Linear Models (GLMs) are an increasingly popular framework for modeling neural spike trains. They have been linked to the theory of stochastic point processes and researchers have used this relation to assess goodness-of-fit using methods from point-process theory, e.g. the time-rescaling theorem. However, high neural firing rates or coarse discretization lead to a breakdown of the assumptions necessary for this connection. Here, we show how goodness-of-fit tests from point-process theory can still be applied to GLMs by constructing equivalent surrogate point processes out of time-series observations. Furthermore, two additional tests based on thinning and complementing point processes are introduced. They augment the instruments available for checking model adequacy of point processes as well as discretized models.
BibTeX
@inproceedings{gerhard2010rescaling,
  author = {Felipe Gerhard and Wulfram Gerstner},
  title = {{Rescaling, thinning or complementing? On goodness-of-fit procedures for point process models and Generalized Linear Models}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2010},
  url = {https://openreview.net/forum?id=Bk-6t8W_bB},
}

Loading related content...

Figure