Multivariate Phase Synchrony and Auditory Attention Decoding for Speech and Music

M. Asjid Tanveer, Jesper Jensen, Zheng‐Hua Tan, Jan ØstergaardBIO 2025 · DOI

This paper, presented at ICBEB 2025, explores the intersection of neural signal processing and cognitive psychology to understand how our brains manage the "cocktail party problem"—the challenge of focusing on one sound source amidst competing noise.

TL;DR

Researchers have established a significant link between how well the brain tracks a target sound (Auditory Attention Decoding) and the cognitive effort required to do so (measured via Circular Omega Complexity and alpha power). Using a deep learning pipeline, the study demonstrates that as neural tracking of a target source improves, global brain synchrony and listening effort decrease, regardless of whether the background "noise" is speech or music.

What Problem Does This Paper Address?

In noisy environments, the brain must perform two distinct but related tasks: Auditory Attention Decoding (AAD)—identifying and "tracking" the target stream—and exerting listening effort to filter out distractors.

While researchers have developed models to decode attention from EEG signals, the relationship between these decoding scores and the physiological cost of listening is not fully understood, especially when the sounds aren't just speech (e.g., listening to music while someone is talking). Traditional measures of listening effort often rely on bivariate phase synchrony, which fails to capture the "full picture" of global brain activity. This paper aims to bridge that gap by using a multivariate approach to measure brain complexity and synchrony.

Methods

The authors employed a sophisticated multi-stage pipeline:

  • Deep Learning AAD: They used an end-to-end deep learning architecture that performs source separation directly on audio envelopes. It uses the scale-invariant source-to-noise ratio (SI-SNR) as a loss function to reconstruct the target audio from the EEG data.
  • Circular Omega Complexity (COC): This is a relatively new multivariate phase synchrony index. Unlike traditional methods that look at pairs of EEG channels, COC provides a global measure of synchrony across multiple channels (specifically six channels in the parietal region: CP1, CP2, PO3, PO4, P1, and P2).
  • Alpha Power: The study also tracked alpha band (8–12.5 Hz) power, a well-established marker of cognitive load and inhibitory processing.
  • Statistical Modeling: Linear Mixed Models (LMMs) were used to analyze the data, allowing the researchers to account for the non-independence of trials within the same subject.

Datasets

The study utilized a dataset involving 35 participants (21 males, 14 females) with normal hearing.

  • The Task: Participants listened to 60-second trials where two audio streams were presented at $\pm30^\circ$ azimuth. They were instructed to attend to one (target) and ignore the other (distractor).
  • Audio Scenarios: The trials covered four combinations: Speech vs. Speech, Speech vs. Music, Music vs. Speech, and Music vs. Music.
  • EEG: 64-channel EEG data was recorded at 512 Hz.

Key Insights and Ideas

  • Target Tracking vs. Distractor Rejection: The most striking finding is that listening effort is primarily driven by how well the brain tracks the target, not how well it ignores the distractor. The AAD scores for the target significantly predicted COC and alpha power, while distractor AAD scores showed almost no correlation.
  • The Effort-Tracking Inverse Relationship: There is a clear inverse relationship: as AAD performance increases (meaning the brain is successfully "locking onto" the target), both COC and alpha power decrease. This suggests that high brain synchrony and alpha power are markers of the brain "working harder" to compensate for poor signal tracking.
  • Music is a Unique Distractor: Participants found the "Music vs. Speech" scenario (where music was the target) the most challenging. Interestingly, in the "Music vs. Music" scenario, individual AAD scores weren't significant predictors of effort, but the combined tracking of both target and distractor was, suggesting a different cognitive strategy when dealing with purely melodic environments.

Key Results

  • AAD Accuracy: The deep learning model significantly outperformed traditional linear models, achieving approximately 80% accuracy across most scenarios, compared to ~65% for the linear version.
  • Statistical Significance: LMMs confirmed a highly significant effect ($p < 0.0001$) of target AAD scores on both COC index and alpha power.
  • Gradient Analysis: The "Speech vs. Speech" condition showed the most significant decrease in COC as AAD scores improved (Gradient: -2.87), indicating that competing speech requires the most dynamic adjustment of neural resources.

Limitations and Future Work

  • Regional Specificity: The study focused exclusively on the parietal region for COC and alpha power. Future research could explore whether other brain regions (like the frontal or temporal lobes) provide different insights into the attention-effort relationship.
  • Subject Demographics: The study was limited to healthy, young participants. The authors note that investigating these relationships in hearing-impaired individuals—who naturally exert much higher listening effort—is a crucial next step.
  • Application in Hearables: The ultimate goal of this research is to inform the development of "cognitively steered" hearing aids. By monitoring both AAD and COC in real-time, future devices could automatically adjust noise-canceling parameters when they detect a user is struggling (high COC) despite trying to attend to a source.

Summary updated

Abstract

This study explores the relationship between auditory attention decoding (AAD), multivariate phase synchrony, and energy (alpha power) in a scenario with competing speech and music, where subjects were asked to attend to a target source in the presence of a distractor. We use an end-to-end deep learning pipeline to extract AAD scores for the target and distracting audio sources and use the circular omega complexity (COC) index, (a recently proposed correlate of listening effort), to extract multivariate phase synchrony from multi-channel electroencephalogram (EEG) signals. The audio sources consist of speech and music combined in four different scenarios for the target and distractor: speech vs. speech, speech vs. music, music vs. speech, and music vs. music. Our results indicate that as the AAD performance decreases, both the COC index and alpha power increase, suggesting increased recruitment of brain regions in conditions characterized by low speech tracking. We use linear mixed models (LMMs) to test the significance of the effect that AAD scores have on COC and alpha power. Our results indicate a significant relationship between target AAD score and COC as well as alpha power, while no significance is seen between distractor AAD score and COC or alpha power.
BibTeX
@inproceedings{tanveer2025multivariate,
  author = {M. Asjid Tanveer and Jesper Jensen and Zheng‐Hua Tan and Jan Østergaard},
  title = {{Multivariate Phase Synchrony and Auditory Attention Decoding for Speech and Music}},
  booktitle = {BIO},
  year = {2025},
  doi = {10.1051/bioconf/202519501001},
}
Figure