Does the brain care about averages? A simple test

Alejandro Tlaie, Katharine Shapcott, Thijs L. van der Plas, James Rowland, Robert Lees, Joshua Keeling, Adam Packer, Paul Tiesinga, Marieke Schölvinck, Martha Nari HavenithOpenReview 2023 · OpenReview

TL;DR

This paper introduces a simple test to evaluate whether trial-averaged neuronal activity truly reflects how the brain processes information. They found that in highly controlled experimental settings, average responses are reliable and predictive of behavior, but in more naturalistic tasks, they are neither, suggesting that other coding mechanisms are at play.

What Problem Does This Paper Address?

Neuroscientists frequently use "trial-averaged" metrics, like tuning curves, to understand neuronal activity. This approach assumes that neuronal responses are consistent enough across repeated trials to form a reliable "average template" that downstream brain regions can use. Furthermore, it's assumed that single-trial responses that closely match this average template are more likely to lead to correct behavior. However, the authors question whether these assumptions hold true in all experimental contexts, especially in more complex, less controlled tasks where stimuli don't repeat perfectly and behavior is more dynamic. They aim to formally test these assumptions to determine when and if trial-averaged responses are genuinely informative for neuronal computation.

Methods

The core of the paper is a novel test based on two assumptions:

  1. Reliability: Single-trial neuronal responses should be consistently similar to the trial-averaged response for the same stimulus condition.
  2. Behavioral Relevance: Single-trial responses that better match the trial-averaged template should lead to more successful behavioral outcomes.

To quantify these, they developed two metrics:

  • Specificity Index: Measures how much more a single-trial response correlates with the "correct" average template (corresponding to the actual stimulus) compared to an "incorrect" average template.
  • Behavioral Relevance Index (Ω): Quantifies how much better-matched single-trial responses predict correct behavior, using the Vargha-Delaney effect size.

They applied these metrics to two distinct datasets.

Datasets

  1. Data Set 1: Two-photon calcium imaging recordings from primary (S1) and secondary (S2) somatosensory cortices in mice. These mice were trained to detect a precisely controlled optogenetic stimulus in S1. This dataset represents a highly controlled experimental setting where the stimulus directly drives neuronal responses.
  2. Data Set 2: High-density electrophysiological recordings from 71 brain areas in mice performing a two-choice contrast discrimination task. This is a less restrictive paradigm where mice used a steering wheel to select gratings of varying contrast.

Key Insights and Ideas

  • The "average" isn't always the brain's story: The study reveals that the relevance of trial-averaged neuronal activity is highly context-dependent.
  • Controlled environments favor averages: In highly controlled experiments (like optogenetic stimulation), single-trial responses are indeed reliable and predictive of behavior, supporting the use of averages.
  • Naturalistic tasks demand dynamic coding: In more complex tasks (like contrast discrimination), average firing rates are neither reliable nor predictive of behavior, suggesting that the brain employs more dynamic coding strategies.
  • A simple test for computational relevance: The proposed Specificity Index and Behavioral Relevance Index offer a practical way for researchers to assess the validity of using trial-averaged metrics in their own work.

Key Results

  • Data Set 1 (Controlled): Both assumptions were largely met. Single-trial responses were reliably correlated with the average template, and better matches predicted correct behavior (Behavioral Relevance Index Ω ≈ 0.86).
  • Data Set 2 (Less Restricted): Neither assumption was met. Single-trial responses were neither reliably specific to the stimulus nor predictive of behavior (Behavioral Relevance Index Ω ≈ 0.51-0.66, considered negligible).
  • Stimulus Difficulty Matters: In Data Set 2, specificity and relevance increased when analyzing only "easy" trials (large contrast differences), suggesting that average firing rates might be more relevant for coarse, rather than fine, stimulus discrimination.
  • PCA didn't save the day: Applying Principal Component Analysis (PCA) to reduce dimensionality in Data Set 2 did not significantly improve stimulus specificity or behavioral relevance, suggesting the issue isn't just about linear dimensionality reduction.

Limitations and Future Work

  • Data Set 2 Complexity: The authors acknowledge that the less restrictive nature of Data Set 2, with its multiple synaptic relays and more complex behavioral demands, might obscure the role of averages.
  • Single Hemisphere Recording: In Data Set 2, recordings were only from one hemisphere, and the authors speculate that including both might yield more informative templates.
  • Alternative Coding Mechanisms: The findings in Data Set 2 suggest that other coding mechanisms, such as joint neuronal dynamics, temporal coding, or population-level interactions, might be more crucial for processing information in complex tasks.
  • Future Directions: The authors encourage researchers to adopt their simple testing framework to assess the computational relevance of trial-averaged metrics across various contexts. They also suggest exploring more dynamic aspects of neuronal activity beyond simple firing rates.

Summary updated

Abstract

Trial-averaged metrics, e.g. tuning curves or population response vectors, are a ubiquitous way of characterizing neuronal activity. But how relevant are such trial-averaged responses to neuronal computation itself? Here we present a simple test to estimate whether average responses reflect aspects of neuronal activity that contribute to neuronal processing. The test probes two assumptions implicitly made whenever average metrics are treated as meaningful representations of neuronal activity: 1. Reliability: Neuronal responses repeat consistently enough across trials that they convey a recognizable reflection of the average response to downstream regions. 2. Behavioural relevance: If a single-trial response is more similar to the average template, it is more likely to evoke correct behavioural responses. We apply this test to two data sets: (1) Two-photon recordings in primary somatosensory cortices (S1 and S2) of mice trained to detect optogenetic stimulation in S1; and (2) Electrophysiological recordings from 71 brain areas in mice performing a contrast discrimination task (Steinmetz et al. 2019, Nature). Under the highly controlled settings of data set 1, both assumptions were largely fulfilled: Single-trial responses were approximately as related to the average as would be expected if they represented discrete, down-sampled versions of the average response. Moreover, better-matched single-trial responses predicted correct behaviour. In contrast, the less restrictive paradigm of data set 2 met neither assumption, with the match between single-trial and average responses being neither reliable nor predictive of behaviour. We conclude that in data set 2, average responses do not seem particularly relevant to neuronal computation, potentially because information is encoded more dynamically when behaviour is less tightly restricted. Most importantly, we encourage researchers to apply this simple test of computational relevance whenever using trial-averaged neuronal metrics, in order to gauge how representative cross-trial averages are in a given context.
BibTeX
@misc{tlaie2023does,
  author = {Alejandro Tlaie and Katharine Shapcott and Thijs L. van der Plas and James Rowland and Robert Lees and Joshua Keeling and Adam Packer and Paul Tiesinga and Marieke Schölvinck and Martha Nari Havenith},
  title = {{Does the brain care about averages? A simple test}},
  year = {2023},
  url = {https://openreview.net/forum?id=XB6fpAQv0J},
  howpublished = {OpenReview},
}

Loading related content...

Figure