Machine learning for MEG during speech tasks

Demetres Kostas, Elizabeth W. Pang, Frank RudziczScientific 2019 · DOI

Abstract

We consider whether a deep neural network trained with raw MEG data can be used to predict the age of children performing a verb-generation task, a monosyllable speech-elicitation task, and a multi-syllabic speech-elicitation task. Furthermore, we argue that the network makes predictions on the grounds of differences in speech development. Previous work has explored taking 'deep' neural networks (DNNs) designed for, or trained with, images to classify encephalographic recordings with some success, but this does little to acknowledge the structure of these data. Simple neural networks have been used extensively to classify data expressed as features, but require extensive feature engineering and pre-processing. We present novel DNNs trained using raw magnetoencephalography (MEG) and electroencephalography (EEG) recordings that mimic the feature-engineering pipeline. We highlight criteria the networks use, including relative weighting of channels and preferred spectro-temporal characteristics of re-weighted channels. Our data feature 92 subjects aged 4-18, recorded using a 151-channel MEG system. Our proposed model scores over 95% mean cross-validation accuracy distinguishing above and below 10 years of age in single trials of un-seen subjects, and can classify publicly available EEG with state-of-the-art accuracy.

Introduction

Understanding the trajectory of human brain development, particularly regarding language and speech, remains a central challenge in neuroscience. Traditional analysis of brain recordings like magnetoencephalography (MEG) or electroencephalography (EEG) typically relies on a heavy "feature engineering" pipeline. In this approach, experts manually select frequency bands (such as alpha or beta waves) and apply spatial filters based on prior knowledge. While effective, this method is susceptible to human bias and may overlook subtle, unknown patterns within the raw data.

Conversely, standard "off-the-shelf" deep learning models often treat brain data as 2D images, failing to account for the unique relationship between sensors (spatial) and neural oscillations (temporal). This research introduces a deep learning architecture designed to learn these features automatically from raw data while remaining grounded in neurophysiological realities.

Methods

The researchers proposed two primary architectures: the Spatial summary Convolutional Neural Network (SCNN) and the Ra-SCNN. The SCNN mimics the traditional neuroscience pipeline by using spatial convolutions to "mix" signals across brain regions before applying temporal filters to extract frequency-based features. The Ra-SCNN augments this with a Recurrent Neural Network (LSTM) and an attention mechanism, allowing the model to focus on high-importance segments of the recording.

The proposed end-to-end and feature-based processing pipeline for MEG and audio data.

To address the challenges of small medical datasets, the team employed cropping augmentation, generating hundreds of "sub-trials" from single recordings. They also utilized activation maximization to interpret the model's decisions, revealing what specific inputs triggered the network's internal filters.

Key Results

The models demonstrated exceptional performance in age classification tasks. In binary classification (distinguishing subjects under 10 from those 10 and older), the SCNN achieved a 95.1% mean cross-validation accuracy.

When tasked with fine-grained prediction across seven distinct age brackets, the models significantly outperformed chance. As shown in the confusion matrices below, most errors were "one-off" (e.g., predicting a 14-year-old as 12), indicating that the model successfully captured the continuous nature of neural development.

Fine-grained age classification confusion matrices for SCNN and Ra-SCNN models.

Beyond age prediction, the study yielded several key insights:

  • Automatic Localization: The model's internal filters naturally focused on the inferior frontal and dorsolateral prefrontal regions—known hubs for language and motor control—without prior instruction.
  • Frequency Proxies: The network identified that younger children (<10 years) showed higher spectral density in lower frequencies (0–14 Hz), while older children exhibited more activity in high-beta and low-gamma ranges (15–22 Hz).
  • Generalization: Tested on a public EEG motor imagery dataset, the SCNN achieved state-of-the-art results (~73% accuracy), proving its utility as a general-purpose tool for electrophysiological data.

Discussion and Future Work

While highly accurate, the current model treats age as a categorical variable. Future iterations aim to move toward regression models for month-level precision. The researchers also suggest developing "general-purpose" early layers by training on massive, combined open-access datasets, which could then be fine-tuned for specific clinical questions. Ultimately, this work paves the way for connecting neural activity more directly to physical articulator movements through complex speech models.

Summary updated

Abstract

We consider whether a deep neural network trained with raw MEG data can be used to predict the age of children performing a verb-generation task, a monosyllable speech-elicitation task, and a multi-syllabic speech-elicitation task. Furthermore, we argue that the network makes predictions on the grounds of differences in speech development. Previous work has explored taking 'deep' neural networks (DNNs) designed for, or trained with, images to classify encephalographic recordings with some success, but this does little to acknowledge the structure of these data. Simple neural networks have been used extensively to classify data expressed as features, but require extensive feature engineering and pre-processing. We present novel DNNs trained using raw magnetoencephalography (MEG) and electroencephalography (EEG) recordings that mimic the feature-engineering pipeline. We highlight criteria the networks use, including relative weighting of channels and preferred spectro-temporal characteristics of re-weighted channels. Our data feature 92 subjects aged 4-18, recorded using a 151-channel MEG system. Our proposed model scores over 95% mean cross-validation accuracy distinguishing above and below 10 years of age in single trials of un-seen subjects, and can classify publicly available EEG with state-of-the-art accuracy.
BibTeX
@inproceedings{kostas2019machine,
  author = {Demetres Kostas and Elizabeth W. Pang and Frank Rudzicz},
  title = {{Machine learning for MEG during speech tasks}},
  booktitle = {Scientific},
  year = {2019},
  doi = {10.1038/s41598-019-38612-9},
}
Figure