Measuring Sequences of Representations with Temporally Delayed Linear Modelling

Yunzhe Liu, Raymond J. Dolan, Hector Luis Penagos-Vargas, Zeb Kurth‐Nelson, Timothy E.j. BehrensbioRxiv 2020 · DOI

TL;DR

This paper introduces TDLM, a novel computational framework for analyzing sequential patterns in neural activity, particularly during rest. By combining non-linear classification with linear temporal modeling, TDLM can uncover and quantify "replay" of neural representations, offering a powerful tool to bridge findings across human and animal neuroscience.

What Problem Does This Paper Address?

Neuroscience has made strides in understanding how the brain represents information during active tasks. However, the rich structure of neural activity during rest, or "off-task" periods, remains less understood. This resting activity is thought to be crucial for various cognitive functions, including memory consolidation and planning, often manifesting as the reactivation of previously encountered information, known as "replay." The challenge lies in developing methods that can reliably detect and quantify these sequential patterns in neural representations from non-invasive human neuroimaging data, while controlling for various confounds. Existing methods often struggle with temporal autocorrelations, spatial correlations between neural representations, and the need to distinguish between forward and backward sequences.

Methods

The core of TDLM is a two-stage linear generative model.

  • Stage 1: Mapping Neural Activity to States: This involves supervised learning to decode neural activity into discrete "states" or representations. For example, using MEG data, classifiers are trained to identify patterns corresponding to specific stimuli (e.g., faces, houses). This stage aims to find the optimal time point for classifier training by maximizing cross-validation accuracy.
  • Stage 2: Quantifying Temporal Structure (Sequenceness): Once states are identified, TDLM uses a two-level General Linear Model (GLM) to quantify the likelihood of sequences.
  • First Level GLM: This step constructs an "empirical transition matrix" by regressing lagged state time courses onto current ones. This matrix captures how likely one state is to follow another with a specific time delay.
  • Second Level GLM: This stage compares the empirical transition matrix to a hypothesized "task-related transition matrix" (representing the sequence of interest). This comparison yields a "sequenceness" score, indicating the degree to which the neural data follows the hypothesized sequence.

TDLM incorporates several key innovations to address confounds:

  • Multiple Linear Regression: Controls for temporal autocorrelations and spurious correlations between states by including all states in the regression model.
  • Controlling for Oscillations: Explicitly models and controls for background neural oscillations (e.g., alpha rhythm) that can inflate false positives.
  • Regularization (L1): Used in classifier training to reduce spatial correlations between state representations, improving sequence detection sensitivity.
  • Permutation Testing: A non-parametric method is used for statistical inference, shuffling state identities to create a null distribution and assess the significance of observed sequenceness. This method is robust and can be adapted for multiple comparisons.

Extensions to TDLM allow for the analysis of multi-step sequences (e.g., A → B → C) and sequences of sequences, enabling the study of hierarchical temporal structures.

Datasets

The paper utilizes two primary datasets:

  • Human MEG Data: Data from participants performing a series of tasks involving visual stimuli and unscrambling sequences. This data was used for training classifiers and for analyzing resting-state activity to detect replay.
  • Rodent Electrophysiology Data: Data from a rat performing a spatial navigation task. This dataset was used to validate TDLM's ability to replicate known phenomena like theta sequences in rodents.

The authors also generated synthetic datasets to simulate various scenarios, including autocorrelated states and spatial correlations, to rigorously test TDLM's performance and validate its statistical inference methods.

Key Insights and Ideas

  • Resting Brain Activity is Structured: The paper strongly supports the idea that neural activity during rest is not random but contains rich, organized sequential patterns that mirror task-related experiences.
  • TDLM Bridges Human and Animal Neuroscience: The framework's ability to analyze both human MEG and rodent electrophysiology data allows for direct cross-species comparisons of sequential neural computations, a significant step towards a unified understanding of cognition.
  • Controlling Confounds is Crucial: The paper highlights how temporal and spatial correlations, as well as neural oscillations, can lead to false positives in sequence detection. TDLM's methods for controlling these confounds are a key contribution.
  • Sequences are More Than Just Transitions: The extensions to TDLM for multi-step sequences and sequences of sequences reveal the potential to uncover hierarchical and complex temporal dynamics in neural representations.
  • Statistical Rigor is Paramount: The emphasis on non-parametric permutation testing and careful control for multiple comparisons underscores the importance of robust statistical inference in this domain.

Key Results

  • TDLM successfully detected sequential reactivation (replay) in human resting-state MEG data, showing strong parallels to rodent hippocampal replay.
  • The method demonstrated its ability to control for confounds like temporal autocorrelation, spatial correlations, and alpha oscillations, leading to more reliable sequence detection.
  • Simulations showed that TDLM's L1 regularization improved sequence detection sensitivity by reducing spatial correlations between state representations.
  • The framework was able to replicate well-known phenomena in rodent data, such as theta sequences, demonstrating its cross-species applicability.
  • The paper provides a detailed methodology for statistical inference, including methods for correcting for multiple comparisons over time lags.

Limitations and Future Work

  • Decoding Accuracy: The performance of TDLM is dependent on the accuracy of the initial state decoding. If states cannot be reliably decoded, sequence detection will be compromised.
  • Assumption of Exchangeability: The permutation tests rely on the assumption that state identities are exchangeable under the null hypothesis. The paper notes that this assumption can be broken in certain scenarios (e.g., due to extensive task training leading to representational overlap), which could lead to false positives. While they haven't found a simulation regime for this, it's a potential concern.
  • Source Localization: While TDLM can identify the temporal onset of sequences, precisely localizing these sequences in the brain remains an area for further development, though the authors suggest combining TDLM with fMRI for improved spatial resolution.
  • Future Work: The authors suggest applying TDLM to fMRI data, exploring its use in on-task sequential reactivation, and investigating its potential for understanding more complex cognitive functions like language, planning, and inference. They also highlight the potential for TDLM to test specific neural predictions from computational models.

Summary updated

Abstract

SUMMARY There are rich structures in off-task neural activity. For example, task related neural codes are thought to be reactivated in a systematic way during rest. This reactivation is hypothesised to reflect a fundamental computation that supports a variety of cognitive functions. Here, we introduce an analysis toolkit (TDLM) for analysing this activity. TDLM combines nonlinear classification and linear temporal modelling to testing for statistical regularities in sequences of neural representations. It is developed using non-invasive neuroimaging data and is designed to take care of confounds and maximize sequence detection ability. The method can be extended to rodent electrophysiological recordings. We outline how TDLM can successfully reveal human replay during rest, based upon non-invasive magnetoencephalography (MEG) measurements, with strong parallels to rodent hippocampal replay. TDLM can therefore advance our understanding of sequential computation and promote a richer convergence between animal and human neuroscience research.
BibTeX
@misc{liu2020measuring,
  author = {Yunzhe Liu and Raymond J. Dolan and Hector Luis Penagos-Vargas and Zeb Kurth‐Nelson and Timothy E.J. Behrens},
  title = {{Measuring Sequences of Representations with Temporally Delayed Linear Modelling}},
  year = {2020},
  doi = {10.1101/2020.04.30.066407},
  howpublished = {bioRxiv},
}
Figure