Simultaneous electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) data during an inner speech task

Foteini Liwicki, Rajkumar Saini, Debashis Das Chakladar, Sumit Rakesh, Vibha Gupta, Marcus Liwicki, Johan G. ErikssonData 2025 · DOI

TL;DR

This paper introduces a novel, publicly available dataset of simultaneously recorded EEG and fMRI data during an "inner speech" task. This multimodal dataset aims to advance research in decoding covert speech, which is crucial for developing assistive communication technologies for individuals with speech impairments.

What Problem Does This Paper Address?

Inner speech, the internal generation of language without overt articulation, is a fundamental cognitive process with significant implications for brain-computer interfaces (BCIs). Developing BCIs that can decode inner speech could revolutionize assistive communication for individuals with severe speech and motor impairments, such as those with ALS or certain neurodevelopmental conditions. However, reliably decoding inner speech is challenging due to the lack of articulatory movements and the complex neural mechanisms involved. Existing datasets often rely on single neuroimaging modalities (either EEG or fMRI), which limits the ability to fully capture the dynamic neural activity associated with inner speech. This paper addresses this gap by providing a synchronized EEG-fMRI dataset, enabling researchers to explore the neural correlates of inner speech with greater detail and develop more sophisticated multimodal analysis techniques.

Methods

The study collected data using a combination of electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) simultaneously.

  • EEG Acquisition: A 64-channel MR-compatible EEG cap (BrainCap MR) was used, sampling at 5 kHz. Electrocardiogram (ECG) was also recorded to help correct for cardioballistic artifacts.
  • fMRI Acquisition: A 3T scanner with a 48-channel headcoil was employed, using an echo-planar imaging (EPI) sequence optimized for whole-brain coverage with a repetition time (TR) of 2 seconds.
  • Artifact Correction: Both EEG and fMRI data underwent artifact correction. For EEG, this included correcting for gradient artifacts (using average artifact subtraction) and cardioballistic artifacts (time-locked to ECG R-peaks). For fMRI, standard preprocessing steps would typically be applied by researchers using the data.
  • Experimental Design: Participants performed an inner speech task where they silently repeated visually presented words (social or numerical categories). Each trial involved a 2-second task period followed by a 12-second rest period.

Datasets

The paper introduces a newly collected dataset:

  • Synchronous EEG and fMRI Dataset on Inner Speech: This dataset, collected from three healthy participants, is publicly available on the OpenNeuro repository (doi:10.18112/openneuro.ds006033.v1.0.0). It is organized according to the Brain Imaging Data Structure (BIDS) format. This dataset complements previous work by the authors, which provided an asynchronous bimodal dataset for inner speech.

Key Insights and Ideas

  • Multimodal Synergy for Inner Speech: The core idea is that combining EEG's high temporal resolution with fMRI's high spatial resolution offers a more comprehensive view of the neural dynamics during inner speech than either modality alone. This is crucial for understanding the rapid neural events and the specific brain regions involved.
  • Benchmarking for Fusion Techniques: The dataset is intended to serve as a valuable benchmark for developing and validating advanced EEG-fMRI fusion techniques. Researchers can use it to test various integration strategies, from model-based approaches to data-driven methods and machine learning algorithms.
  • Advancing BCIs for Communication Impairments: By providing a resource for decoding inner speech, this dataset directly supports the development of more effective assistive technologies for individuals who cannot communicate through conventional means.

Key Results

The paper's primary contribution is the creation and release of the dataset itself, rather than presenting specific quantitative results from analyzing it. The "results" are the characteristics of the dataset:

  • Simultaneous Recording: The dataset captures EEG and fMRI data concurrently, preserving the temporal relationship between neural signals.
  • Task-Specific Data: It includes data specifically from an inner speech task, allowing for the investigation of neural correlates of covert verbalization.
  • Rich Modalities: It offers both high-temporal resolution (EEG) and high-spatial resolution (fMRI) data, providing complementary information.
  • Publicly Available: The dataset is accessible via OpenNeuro, promoting open science and enabling broader research participation.

Limitations and Future Work

  • Limited Participants: The dataset was collected from only three healthy participants. This small sample size limits the generalizability of findings derived from it and may pose challenges for training complex machine learning models without further data augmentation or transfer learning.
  • Missing Data: EEG recordings for subject 01, session 01, and an interrupted recording for subject 02, session 01, are noted as unavailable due to technical reasons.
  • Non-Native Speakers: Participants were not native English speakers, which might influence the nature of inner speech production and its neural correlates compared to native speakers.
  • Future Work: The authors explicitly state the dataset is intended to support research in:
  • Inner speech processing.
  • Multimodal neuroimaging.
  • EEG-fMRI fusion techniques.
  • BCI-driven speech prosthesis development.
  • Studies on consciousness and the default mode network.
  • Machine learning models for inner speech recognition.

Summary updated

Abstract

Inner speech, or covert speech, refers to the internal generation of language without overt articulation. Decoding inner speech has significant implications for brain-computer interfaces (BCIs), particularly for assistive communication in individuals with speech and motor impairments. To facilitate research in this area, we introduce a publicly available dataset comprising simultaneously recorded electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) data during inner speech production. Data were collected from three healthy, right-handed participants performing an inner speech task. The task involved silent repetition of visually presented words belonging to either a social or numerical category. The experiment consisted of 40 trials per word, with eight unique words and starts with a fixation period of two seconds. Stimuli were displayed for two seconds at the beginning of each session, followed by a 12-second rest period to allow hemodynamic responses to return to baseline. Participants were instructed to remain still and avoid movements to minimize artifacts. EEG was recorded using a 64-channel MR-compatible cap (BrainCap MR, EasyCap GmbH) at a 5 kHz sampling rate. Electrocardiogram (ECG) signals were simultaneously acquired using an additional electrode placed on the trapezius muscle to facilitate cardioballistic artifact correction. Gradient and cardioballistic artifacts were corrected using BrainVision Analyzer software. Functional MRI data were acquired using a 3T scanner with a 48-channel headcoil, and an echo-planar imaging (EPI) sequence optimized for whole-brain coverage. The repetition time (TR) was 2 s. High-resolution anatomical T1-weighted images were also acquired for structural reference. The dataset is publicly available in the OpenNeuro repository. The aim of this dataset is to provide a resource for studying inner speech processing, multimodal neuroimaging, EEG-fMRI fusion techniques, and BCI-driven speech prosthesis development.
BibTeX
@inproceedings{liwicki2025simultaneous,
  author = {Foteini Liwicki and Rajkumar Saini and Debashis Das Chakladar and Sumit Rakesh and Vibha Gupta and Marcus Liwicki and Johan G. Eriksson},
  title = {{Simultaneous electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) data during an inner speech task}},
  booktitle = {Data},
  year = {2025},
  doi = {10.1016/j.dib.2025.112258},
}
Figure