Measuring Information-Transfer Delays

Michael Wibral, Nicolae Cristian Pampu, Viola Priesemann, Felix Siebenhühner, Hannes Seiwert, Michael Lindner, Joseph T. Lizier, Raúl VicentePLoS 2013 · DOI

Abstract: In complex networks such as gene networks, traffic systems, or brain circuits, it is important to understand how long it takes for different parts of the network to effectively influence one another. This paper proposes a robust estimator for neuronal interaction delays rooted in an information-theoretic framework, allowing for model-free exploration. By extending transfer entropy to account for delayed source-target interactions while retaining conditioning on the target state at the immediately previous time step, the authors introduce Self-Prediction Optimal Transfer Entropy ($TE_{SPO}$). This method is proven to identify interaction delays accurately, adhering to Wiener's principle of causality. Performance is demonstrated through numerical simulations and local field potential recordings from the turtle brain.

Introduction

In complex systems like the brain, the stock market, or a power grid, events do not happen instantaneously. An event in one part of a network takes time to travel and influence another. Measuring these "information-transfer delays" is critical for understanding network function. However, traditional methods for estimating these delays often suffer from modeling assumptions or mathematical flaws that lead to incorrect interpretations of directed interactions.

The "Self-Prediction" Trap

Researchers frequently use Transfer Entropy (TE) to measure directed information flow. TE is based on Wiener’s principle of causality: $X$ causes $Y$ if the past of $X$ helps predict the future of $Y$ better than the past of $Y$ alone.

When delays are present, previous implementations of "delayed TE" often shifted the observation windows for both the source ($X$) and the target ($Y$) together. This approach violates Wiener’s principle because it fails to account for the state of $Y$ immediately before the event being predicted. By ignoring the target's most recent history, the estimator may "borrow" predictive power from the source that actually belongs to the target's own memory, leading to false positives or incorrect delay estimates.

Conceptual framework of $TE_{SPO}$. Panel A illustrates the relationship between source $X$ and target $Y$ for different delay parameters $u$ relative to the true delay $\delta$. Panel B shows how the $TE$ curve peaks when $u$ matches the true delay $\delta$.

Methods: Self-Prediction Optimal Transfer Entropy ($TE_{SPO}$)

The authors introduce Self-Prediction Optimal Transfer Entropy ($TE_{SPO}$) to resolve this flaw.

  • The Innovation: Unlike previous versions, $TE_{SPO}$ keeps the target's conditioning fixed on the state immediately preceding the prediction point ($Y_{t-1}$) while scanning different delays ($u$) for the source ($X_{t-u}$).
  • The Proof: The authors provide a formal mathematical proof (Theorem 1) showing that $TE_{SPO}$ is maximal only when the delay parameter $u$ matches the true physical delay $\delta$.
  • Implementation: The method uses the Kraskov-Stögbauer-Grassberger (KSG) estimator for model-free analysis of continuous data and is integrated into the open-source TRENTOOL MATLAB toolbox.

Validation and Results

The method was validated across a hierarchy of complexity, from simulations to biological data:

  1. Stochastic and Deterministic Simulations: $TE_{SPO}$ outperformed alternatives like Momentary Information Transfer (MIT), particularly when the source had its own internal memory. It successfully identified delays in non-linear chaotic systems and complex network architectures like rings and bidirectional couplings.
  2. Biological Ground Truth: Using Local Field Potentials (LFP) from the turtle brain, the authors reconstructed delays in a visual processing pathway. The sum of measured delays for Light $\to$ Retina ($30\text{ ms}$) and Retina $\to$ Tectum ($12\text{ ms}$) was $42\text{ ms}$, remarkably close to the directly measured Light $\to$ Tectum delay of $44\text{ ms}$ (a $4.5\%$ error).
  3. Robustness: The estimator remained accurate even with observation noise accounting for up to $9\%$ of total signal variance and could detect multiple delays in systems with feedback loops.

Conclusion

$TE_{SPO}$ provides a robust, model-free framework for measuring interaction delays in dynamic networks. By strictly adhering to Wiener's principle of causality, it avoids the self-prediction errors that plague other methods. While it requires stationary data and sufficient samples for convergence, its ability to handle non-linearities and feedback makes it a versatile tool for neuroscience and beyond.

Summary updated

Abstract

In complex networks such as gene networks, traffic systems or brain circuits it is important to understand how long it takes for the different parts of the network to effectively influence one another. In the brain, for example, axonal delays between brain areas can amount to several tens of milliseconds, adding an intrinsic component to any timing-based processing of information. Inferring neural interaction delays is thus needed to interpret the information transfer revealed by any analysis of directed interactions across brain structures. However, a robust estimation of interaction delays from neural activity faces several challenges if modeling assumptions on interaction mechanisms are wrong or cannot be made. Here, we propose a robust estimator for neuronal interaction delays rooted in an information-theoretic framework, which allows a model-free exploration of interactions. In particular, we extend transfer entropy to account for delayed source-target interactions, while crucially retaining the conditioning on the embedded target state at the immediately previous time step. We prove that this particular extension is indeed guaranteed to identify interaction delays between two coupled systems and is the only relevant option in keeping with Wiener's principle of causality. We demonstrate the performance of our approach in detecting interaction delays on finite data by numerical simulations of stochastic and deterministic processes, as well as on local field potential recordings. We also show the ability of the extended transfer entropy to detect the presence of multiple delays, as well as feedback loops. While evaluated on neuroscience data, we expect the estimator to be useful in other fields dealing with network dynamics.
BibTeX
@inproceedings{wibral2013measuring,
  author = {Michael Wibral and Nicolae Cristian Pampu and Viola Priesemann and Felix Siebenhühner and Hannes Seiwert and Michael Lindner and Joseph T. Lizier and Raúl Vicente},
  title = {{Measuring Information-Transfer Delays}},
  booktitle = {PLoS},
  year = {2013},
  doi = {10.1371/journal.pone.0055809},
}
Figure