TL;DR
This paper introduces a novel, publicly available dataset of simultaneously recorded EEG and fMRI data during an "inner speech" task. This multimodal dataset aims to advance research in decoding covert speech, which is crucial for developing assistive communication technologies for individuals with speech impairments.
What Problem Does This Paper Address?
Inner speech, the internal generation of language without overt articulation, is a fundamental cognitive process with significant implications for brain-computer interfaces (BCIs). Developing BCIs that can decode inner speech could revolutionize assistive communication for individuals with severe speech and motor impairments, such as those with ALS or certain neurodevelopmental conditions. However, reliably decoding inner speech is challenging due to the lack of articulatory movements and the complex neural mechanisms involved. Existing datasets often rely on single neuroimaging modalities (either EEG or fMRI), which limits the ability to fully capture the dynamic neural activity associated with inner speech. This paper addresses this gap by providing a synchronized EEG-fMRI dataset, enabling researchers to explore the neural correlates of inner speech with greater detail and develop more sophisticated multimodal analysis techniques.
Methods
The study collected data using a combination of electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) simultaneously.
- EEG Acquisition: A 64-channel MR-compatible EEG cap (BrainCap MR) was used, sampling at 5 kHz. Electrocardiogram (ECG) was also recorded to help correct for cardioballistic artifacts.
- fMRI Acquisition: A 3T scanner with a 48-channel headcoil was employed, using an echo-planar imaging (EPI) sequence optimized for whole-brain coverage with a repetition time (TR) of 2 seconds.
- Artifact Correction: Both EEG and fMRI data underwent artifact correction. For EEG, this included correcting for gradient artifacts (using average artifact subtraction) and cardioballistic artifacts (time-locked to ECG R-peaks). For fMRI, standard preprocessing steps would typically be applied by researchers using the data.
- Experimental Design: Participants performed an inner speech task where they silently repeated visually presented words (social or numerical categories). Each trial involved a 2-second task period followed by a 12-second rest period.
Datasets
The paper introduces a newly collected dataset:
- Synchronous EEG and fMRI Dataset on Inner Speech: This dataset, collected from three healthy participants, is publicly available on the OpenNeuro repository (doi:10.18112/openneuro.ds006033.v1.0.0). It is organized according to the Brain Imaging Data Structure (BIDS) format. This dataset complements previous work by the authors, which provided an asynchronous bimodal dataset for inner speech.
Key Insights and Ideas
- Multimodal Synergy for Inner Speech: The core idea is that combining EEG's high temporal resolution with fMRI's high spatial resolution offers a more comprehensive view of the neural dynamics during inner speech than either modality alone. This is crucial for understanding the rapid neural events and the specific brain regions involved.
- Benchmarking for Fusion Techniques: The dataset is intended to serve as a valuable benchmark for developing and validating advanced EEG-fMRI fusion techniques. Researchers can use it to test various integration strategies, from model-based approaches to data-driven methods and machine learning algorithms.
- Advancing BCIs for Communication Impairments: By providing a resource for decoding inner speech, this dataset directly supports the development of more effective assistive technologies for individuals who cannot communicate through conventional means.
Key Results
The paper's primary contribution is the creation and release of the dataset itself, rather than presenting specific quantitative results from analyzing it. The "results" are the characteristics of the dataset:
- Simultaneous Recording: The dataset captures EEG and fMRI data concurrently, preserving the temporal relationship between neural signals.
- Task-Specific Data: It includes data specifically from an inner speech task, allowing for the investigation of neural correlates of covert verbalization.
- Rich Modalities: It offers both high-temporal resolution (EEG) and high-spatial resolution (fMRI) data, providing complementary information.
- Publicly Available: The dataset is accessible via OpenNeuro, promoting open science and enabling broader research participation.
Limitations and Future Work
- Limited Participants: The dataset was collected from only three healthy participants. This small sample size limits the generalizability of findings derived from it and may pose challenges for training complex machine learning models without further data augmentation or transfer learning.
- Missing Data: EEG recordings for subject 01, session 01, and an interrupted recording for subject 02, session 01, are noted as unavailable due to technical reasons.
- Non-Native Speakers: Participants were not native English speakers, which might influence the nature of inner speech production and its neural correlates compared to native speakers.
- Future Work: The authors explicitly state the dataset is intended to support research in:
- Inner speech processing.
- Multimodal neuroimaging.
- EEG-fMRI fusion techniques.
- BCI-driven speech prosthesis development.
- Studies on consciousness and the default mode network.
- Machine learning models for inner speech recognition.
Summary updated