Abstract
This study establishes a theoretical framework for the scaling of information-theoretic measures in neural spike trains as time resolution ($\Delta t$) approaches the continuous-time limit. By analyzing various neuron models, including renewal processes and biophysical integrate-and-fire models, the authors derive analytical scaling laws for entropy rate, statistical complexity, and excess entropy. They identify a universality class where informational properties depend primarily on firing variability rather than specific biophysical mechanisms.
Introduction
In computational neuroscience, neural activity is typically discretized into time bins of size $\Delta t$. However, the choice of $\Delta t$ is often arbitrary and significantly impacts the resulting information measures. Large bins obscure timing precision, while excessively small bins lead to undersampling issues in numerical estimation. This research addresses the fundamental behavior of these measures in the limit $\Delta t \to 0$, seeking to understand the intrinsic computation of neurons—how they generate, store, and transmit information.
Methods
The authors utilize computational mechanics, representing stochastic processes through "causal states" or $\epsilon$-machines. This approach allows for a rigorous decomposition of information. The study examines several model classes:
- Renewal Processes: Characterized by independent interspike intervals (ISIs).
- Alternating Renewal Processes (ARPs): Models with switching between two ISI distributions.
- Biophysical Models: Noisy (NIF), Leaky (LIF), and Quadratic (QIF) Integrate-and-Fire neurons.
Analytical derivations were validated using high-fidelity simulations of $10^5$ ISIs to ensure robust estimation of information measures across varying time resolutions.
Information Anatomy
A central contribution of the work is the "information anatomy" of neural activity. This framework decomposes the information in a single time bin into distinct components based on their relationship to the past and future.
As shown in Figure 1, the total entropy $H[X_0]$ is partitioned into components such as ephemeral information ($r_\mu$), which is randomness that does not persist; bound information ($b_\mu$), which is actively stored and useful for future prediction; and enigmatic information ($q_\mu$), representing the co-information shared between the past, present, and future.
Scaling Laws and Convergence
The analysis reveals critical differences in how these measures scale with $\Delta t$. While the entropy rate ($h_\mu$) and statistical complexity ($C_\mu$) diverge logarithmically as resolution increases, the excess entropy ($E$)—representing the total predictability of the process—converges to a stable, finite value.
Figure 4 illustrates these scaling behaviors. The divergence of $C_\mu$ and the entropy rate per unit time ($h_\mu / \Delta t$) suggests they are resolution-dependent descriptions of randomness and memory. In contrast, the convergence of $E$ indicates it is a more robust physical measure for comparing different neural systems.
Universality and Model Selection
A striking finding is the emergence of a "universality class." When information measures are plotted against the Coefficient of Variation ($C_V$), different biophysical models (NIF, LIF, QIF) follow identical curves at high time resolutions. This implies that the specific mechanism of spike generation (e.g., linear vs. quadratic leak) is less important than the overall statistics of the spike intervals.
Furthermore, these scaling "signatures" serve as a model selection tool. For instance, neurons with Gamma-distributed ISIs or those following ARPs deviate from the integrate-and-fire universality curves, allowing researchers to distinguish between underlying mathematical descriptions of real biological data.
Conclusion
This work provides the first comprehensive look at the continuous-time limit of neural information measures. While the assumption of renewal processes covers many cases, future work must address long-range dependencies and adaptation. The difficulty in estimating bound information for high-variability neurons suggests a need for "binless" estimators in practical applications. Extending this framework to dynamic, time-varying stimuli will further test the robustness of the discovered universality.
Summary updated