MAD-EEG: an EEG dataset for decoding auditory attention to a target instrument in polyphonic music

Giorgia Cantisani, Gabriel Trégoat, Slim Essid, Gaël Richard

TL;DR

This paper introduces MAD-EEG, the first freely available EEG dataset specifically designed for decoding auditory attention to a target instrument within complex polyphonic music. By using realistic music stimuli and a novel experimental setup, MAD-EEG enables single-trial attention decoding, paving the way for more nuanced research into how our brains process music and selectively attend to specific sounds.

What Problem Does This Paper Address?

Understanding how humans selectively attend to specific sound sources in a noisy environment is a long-standing challenge in neuroscience and signal processing. While significant progress has been made in decoding auditory attention for speech (e.g., identifying which speaker someone is listening to in a multi-talker scenario), this problem is considerably more complex in the realm of music. Polyphonic music, with its rich tapestry of interwoven instruments and melodies, presents a much greater challenge for auditory attention decoding. Existing music-related EEG datasets are not designed for this specific task, often focusing on broader aspects like emotional response or rhythm, or using highly artificial stimuli. This gap limits research into how we pinpoint and follow a particular instrument's contribution within a musical piece, a skill fundamental to music perception and enjoyment.

Methods

The core contribution of this paper is the creation and validation of the MAD-EEG dataset. The authors designed a novel experimental protocol to collect 20-channel EEG signals from 8 subjects. Participants listened to polyphonic music excerpts (mixtures of 2-3 instruments) played through speakers, a departure from typical headphone-based experiments. Crucially, they were instructed to focus on a specific target instrument within each mix.

To validate the dataset's utility for attention decoding, the researchers employed a stimulus reconstruction approach, a technique previously successful in speech research. This involves training subject-specific linear regression models to map EEG signals to the spectrogram of the audio stimulus. The key idea is that if the EEG accurately tracks attention, the reconstructed audio representation should be more similar to the attended instrument's sound than to unattended ones.

Datasets

The paper introduces the MAD-EEG dataset, a new, freely available resource. It comprises:

  • 20-channel EEG signals from 8 subjects.
  • Audio stimuli consisting of polyphonic music mixes (solos, duets, trios) from pop and classical genres.
  • Stimuli were varied in terms of number of instruments, spatial rendering (monophonic/stereo), musical genre, and theme.
  • Each stimulus was presented repetitively (4 trials) to allow for both single-trial and averaging-based analysis.
  • Subjects provided self-assessments of their attention levels after each section.

This dataset is distinct from existing music-related EEG datasets (like NMED-H, NMED-T, NMED-RP, DEAP, OpenMIIR) because it specifically targets the decoding of attention to a single instrument within a complex musical texture, using realistic, non-artificial stimuli.

Key Insights and Ideas

  • Realistic Music Stimuli are Crucial: The MAD-EEG dataset's use of naturalistic polyphonic music, rather than simplified or artificial musical structures, is vital for studying attention decoding in real-world listening scenarios.
  • Speaker-Based Playback Matters: Presenting music through speakers, rather than headphones, introduces spatial cues that are inherent to real-world listening and can influence auditory attention.
  • Single-Trial Decoding is Possible: The dataset's design, with repetitive stimulus presentation, supports the challenging task of decoding attention from single EEG trials, moving beyond traditional averaging methods.
  • EEG Tracks Attended Instrument: The validation experiment demonstrates that EEG signals contain information that allows for distinguishing the attended instrument from unattended ones in a musical mix.

Key Results

The validation experiment showed promising results:

  • A stimulus reconstruction algorithm, applied to the MAD-EEG data, was able to reconstruct audio representations that were significantly more correlated with the attended instrument than with the unattended instrument in duet mixtures.
  • Specifically, in over 78% of duet trials, the correlation with the attended instrument was higher than with the unattended one.
  • The statistical significance of this finding was very high (p < 10-27 using a Wilcoxon test, and p < 10-15 using a randomization test).
  • The correlation coefficients obtained were comparable to those reported for speech spectrogram reconstruction, indicating the quality of the EEG data and the effectiveness of the approach.

Limitations and Future Work

Limitations: * The dataset currently includes a relatively small number of subjects (8). * While the stimuli were varied, further expansion in terms of musical complexity and genre could be beneficial. * The validation experiment used a relatively simple linear regression model; more complex models could potentially yield even better decoding performance.

Future Work: * The authors plan to extend the dataset by increasing the number of EEG recordings and introducing more variants in the stimuli and behavioral data. * They aim to explore more sophisticated attention decoding models. * The dataset is intended to facilitate research into other EEG-driven Music Information Retrieval (MIR) tasks, such as selective source separation, music generation, and transcription.

Summary updated

Abstract

We present MAD-EEG, a new, freely available dataset for studying EEG-based auditory attention decoding considering the challenging case of subjects attending to a target instrument in polyphonic music. The dataset represents the first music-related EEG dataset of its kind, enabling, in particular, studies on single-trial EEG-based attention decoding, while also opening the path for research on other EEG-based music analysis tasks. MAD-EEG has so far collected 20-channel EEG signals recorded from 8 subjects listening to solo, duo and trio music excerpts and attending to one pre-specified instrument. The proposed experimental setting differs from the ones previously considered as the stimuli are polyphonic and are played to the subject using speakers instead of headphones. The stimuli were designed considering variations in terms of number and type of instruments in the mixture, spatial rendering, music genre and melody that is played. Preliminary results obtained with a state-of-the-art stimulus reconstruction algorithm commonly used for speech stimuli show that the audio representation reconstructed from the EEG response is more correlated with that of the attended source than with the one of the unattended source, proving the dataset to be suitable for such kind of studies.
BibTeX
@inproceedings{cantisani2019mad,
  author = {Giorgia Cantisani and Gabriel Trégoat and Slim Essid and Gaël Richard},
  title = {{MAD-EEG: an EEG dataset for decoding auditory attention to a target instrument in polyphonic music}},
  booktitle = {SMM},
  year = {2019},
  doi = {10.21437/smm.2019-11},
}
Figure