Trial-by-trial predictions of subjective time from human brain activity

Maxine T. Sherman, Zafeirios Fountas, Anil K. Seth, Warrick RoseboomPLOS Computational Biology 2022 · DOI

TL;DR

This paper proposes that our subjective experience of time is constructed by accumulating "salient events" detected in our sensory processing hierarchy. Using fMRI data, they demonstrate that activity in the visual cortex, but not auditory or somatosensory cortices, can predict trial-by-trial biases in how long people perceive events to last, suggesting a modality-specific neural basis for subjective time.

What Problem Does This Paper Address?

Our perception of time isn't a constant, objective measure like a clock. Instead, it's subjective and can be influenced by various factors, leading to experiences like "time flying when you're having fun." While previous research has shown that simple stimulus properties (like brightness or speed) can alter time perception, understanding how this works with complex, naturalistic scenes has been challenging. This paper aims to bridge this gap by proposing a unified computational model that explains subjective time perception by tracking changes in neural activity across sensory processing hierarchies, specifically focusing on how these changes relate to our subjective biases in duration estimation.

Methods

The researchers employed a multi-layered approach:

  • fMRI Data Acquisition: They recorded brain activity (BOLD signals) from 40 participants while they watched silent, naturalistic videos (8-24 seconds long) and estimated their duration.
  • "Salient Event" Detection: They defined "salient events" as significant changes in neural activity within specific brain regions (visual, auditory, and somatosensory cortices) over time. This was quantified using two metrics: Euclidean distance and signed difference between consecutive time points. A dynamic attention threshold was used to identify these salient events.
  • Hierarchical Models: They built three-layer perceptual hierarchies for visual, auditory, and somatosensory cortices. The visual hierarchy was the primary focus, while auditory and somatosensory served as control models.
  • Model-Based Prediction: Accumulated salient events from each layer were used to predict presented video durations (clock time) via support vector regression. Crucially, they then converted these predictions into "normalized bias" (a measure of subjective over- or under-estimation) to compare with human behavioral biases.
  • Confirmatory and Exploratory Analyses: A pre-registered (confirmatory) analysis used Euclidean distance and pooled human data. An exploratory analysis used signed difference and analyzed data on a participant-by-participant basis to assess robustness and capture scene-specific effects.

Datasets

The study used a dataset of naturalistic, silent videos of city and office scenes, previously used in related work [6]. The fMRI data collected from 40 participants during the experiment forms the core dataset for their brain-based modeling.

Key Insights and Ideas

  • Subjective Time from Sensory Dynamics: The core idea is that our subjective experience of time isn't about a dedicated "time module" but rather emerges from the ongoing processing of sensory information. The rate and nature of changes in sensory input, as reflected in neural activity, shape our perception of duration.
  • "Salient Events" as the Currency: The concept of "salient events" – significant shifts in neural activity – provides a unifying measure across different levels of sensory processing. These events, rather than conscious perception of specific features, are proposed to be the building blocks of subjective time.
  • Modality Specificity: The finding that only the visual cortex model could predict subjective duration biases (but not clock time) is critical. This suggests that for visual stimuli, the specific way the visual system processes incoming information is what dictates our subjective experience of time, and this isn't simply a general property of any dynamic neural activity.
  • Signed Difference as a Better Predictor: The exploratory analysis using "signed difference" (which can capture prediction errors) was more successful in predicting both subjective biases and the effect of scene type (city vs. office) compared to Euclidean distance. This hints at a potential role for predictive coding mechanisms in time perception.

Key Results

  • Visual Cortex Predicts Subjective Bias: The model trained on visual cortex activity successfully predicted trial-by-trial subjective duration biases (p = 0.015). Control models using auditory or somatosensory cortex activity did not.
  • Clock Time Prediction is Trivial: All models (visual, auditory, somatosensory) could trivially predict the actual duration of the videos (clock time) with high accuracy (r > 0.93). This highlights that predicting subjective bias, not just duration, is the key challenge.
  • Scene Type Matters (with Signed Difference): Using the "signed difference" metric, the visual cortex model could also predict the effect of scene type on duration bias (city scenes felt longer), which was not achieved with the Euclidean distance metric in the confirmatory analysis.
  • Robustness: The findings were robust across a range of parameter values for defining "salient events," suggesting the core mechanism is not overly sensitive to specific thresholds.

Limitations and Future Work

  • Stimulus Specificity: The study focused on silent visual stimuli. Future work should investigate if similar principles apply to auditory or multisensory contexts, and whether combining information from different sensory modalities is necessary for more complex scenarios.
  • "Salience" Definition: The definition of "salience" is based on neural activity changes, not necessarily psychological salience or surprise. Further research could explore the relationship between these concepts.
  • Simplified Hierarchy: The visual hierarchy used was a simplified model. Expanding the set of regions included could potentially improve model performance.
  • Mechanistic Basis: While the results support a link between predictive processing and time perception (especially with the signed difference metric), the authors emphasize they are theory-neutral regarding the precise computational mechanisms. Future work could further investigate this connection.

Summary updated

BibTeX
@article{sherman2022trial,
  author = {Maxine T. Sherman and Zafeirios Fountas and Anil K. Seth and Warrick Roseboom},
  title = {{Trial-by-trial predictions of subjective time from human brain activity}},
  journal = {PLOS Computational Biology},
  year = {2022},
  doi = {10.1371/journal.pcbi.1010223},
}
Figure