Tracking cortical entrainment in neural activity: auditory processes in human temporal cortex

Andrew Thwaites, Ian Nimmo-Smith, Elisabeth Fonteneau, Roy D. Patterson, Paula Buttery, William D. Marslen-WilsonFrontiers Comput. Neurosci 2015 · OpenReview

Abstract

A primary objective for cognitive neuroscience is to identify how features of the sensory environment are encoded in neural activity. Current auditory models of loudness perception can be used to make detailed predictions about the neural activity of the cortex as an individual listens to speech. We used two such models (loudness-sones and loudness-phons), varying in their psychophysiological realism, to predict the instantaneous loudness contours produced by 480 isolated words. These two sets of 480 contours were used to search for electrophysiological evidence of loudness processing in whole-brain recordings of electro- and magneto-encephalographic (EMEG) activity, recorded while subjects listened to the words. The technique identified a bilateral sequence of loudness processes, predicted by the more realistic loudness-sones model, that begin in auditory cortex at ~80 ms and subsequently reappear, tracking progressively down the superior temporal sulcus (STS) at lags from 230 to 330 ms. The technique was then extended to search for regions sensitive to the fundamental frequency (F0) of the voiced parts of the speech. It identified a bilateral F0 process in auditory cortex at a lag of ~90 ms, which was not followed by activity in STS. The results suggest that loudness information is being used to guide the analysis of the speech stream as it proceeds beyond auditory cortex down STS toward the temporal pole.

Introduction

When listening to speech, the human brain "entrains" to specific acoustic features, such as the rhythmic envelope or pitch variations. While this phenomenon of cortical entrainment is well-documented, the specific neural algorithms that transform raw acoustic input into internal representations remain partially understood. This study addresses whether psychophysically realistic models of hearing—those accounting for cochlear compression—better predict brain activity than simpler models, and maps the spatio-temporal progression of loudness and fundamental frequency (F0) processing as speech information moves from the auditory cortex to higher-level language centers.

Methods

The researchers recorded brain activity from 20 native British English speakers using a combined Electroencephalography (EEG) and Magnetoencephalography (MEG) approach, providing high temporal resolution and spatial localization. Minimum Norm Estimation (MNE) was used to map these external sensor signals back to approximately 5,000 specific "sources" or vertices on the cortical surface.

The study compared three candidate models for 480 isolated words and phrases:

  1. Loudness-sones: A psychophysically realistic model incorporating cochlear compression.
  2. Loudness-phons: A simpler, log-based loudness model.
  3. F0 model: An autocorrelation-based model to track the fundamental frequency of voiced speech segments.
Comparison of acoustic pressure and candidate model outputs for loudness (sones and phons) and fundamental frequency (F0).

A "spatio-temporal searchlight" approach was employed, where model predictions were cross-correlated with actual brain activity at every cortical point across time lags from 0 to 700 ms. Statistical significance was established by comparing "matched" pairs (brain response to word X vs. model for word X) against "mismatched" pairs (brain response to word X vs. model for word Y).

Schematic of the cross-correlation analysis between candidate model outputs and neural activity.

Results

The analysis revealed that the Loudness-sones model significantly outperformed the simpler Loudness-phons model, indicating that the brain's internal representation of speech is shaped by the non-linear mechanics of the cochlea.

Key findings include:

  • Early Tracking: Both loudness and F0 showed strong bilateral entrainment in the auditory cortex (Heschl’s Gyrus) at lags of 75–90 ms.
  • The STS "Loudness Highway": Unlike F0, loudness representations reappeared at later lags (210–330 ms) and traveled progressively down the Superior Temporal Sulcus (STS) toward the temporal pole.
  • Spatial Progression: This late-stage loudness activity moved anteriorly along the STS at a velocity of approximately 0.3 to 0.4 m/s, consistent with known neural propagation speeds.
  • Feature Segregation: The divergence in processing paths suggests that while both features are initially tracked in the auditory cortex, loudness information is specifically "handed off" to areas involved in higher-level linguistic analysis.

Discussion

The results demonstrate that EMEG can reliably detect subtle tracking signals across the whole brain, surviving conservative multiple-comparison testing ($p \approx 3 \times 10^{-13}$). The study highlights the importance of biological realism in auditory modeling. While the absolute correlation values remain low (often < 0.3) due to the signal-to-noise ratio of non-invasive electrophysiology, the statistical robustness of the findings is clear. Future work may extend this model-matching technique to other phonetic features or investigate how these instantaneous representations integrate into higher-order perceptual models.

Summary updated

Abstract

p>A primary objective for cognitive neuroscience is to identify how features of the sensory environment are encoded in neural activity. Current auditory models of loudness perception can be used to make detailed predictions about the neural activity of the cortex as an individual listens to speech. We used two such models (loudness-sones and loudness-phons), varying in their psychophysiological realism, to predict the instantaneous loudness contours produced by 480 isolated words. These two sets of 480 contours were used to search for electrophysiological evidence of loudness processing in whole-brain recordings of electro- and magneto-encephalographic (EMEG) activity, recorded while subjects listened to the words. The technique identified a bilateral sequence of loudness processes, predicted by the more realistic loudness-sones model, that begin in auditory cortex at ~80 ms and subsequently reappear, tracking progressively down the superior temporal sulcus (STS) at lags from 230 to 330 ms. The technique was then extended to search for regions sensitive to the fundamental frequency (F0) of the voiced parts of the speech. It identified a bilateral F0 process in auditory cortex at a lag of ~90 ms, which was not followed by activity in STS. The results suggest that loudness information is being used to guide the analysis of the speech stream as it proceeds beyond auditory cortex down STS toward the temporal pole.</p>
BibTeX
@inproceedings{thwaites2015tracking,
  author = {Andrew Thwaites and Ian Nimmo-Smith and Elisabeth Fonteneau and Roy D. Patterson and Paula Buttery and William D. Marslen-Wilson},
  title = {{Tracking cortical entrainment in neural activity: auditory processes in human temporal cortex}},
  booktitle = {Frontiers Comput. Neurosci},
  year = {2015},
  url = {https://openreview.net/forum?id=NvxvqFIHvH},
}
Figure