Abstract
Auditory Attention Detection (AAD) is a critical technology for developing "smart" hearing aids that can isolate a target speaker in noisy environments. However, existing methods often struggle with short decision windows and high computational costs. We present DBPNet, a novel dual-branch parallel network that fuses temporal features from a Transformer-based branch with frequency features from a 3D residual convolutional branch. DBPNet achieves a 20.4% relative accuracy improvement on 0.1s windows while significantly reducing parameter count by 91 times compared to previous state-of-the-art models, making it suitable for real-time, low-power applications.
Introduction
In complex acoustic environments—the classic "cocktail party"—the human brain can effortlessly tune into one voice while filtering out others. For individuals with hearing impairments, this task is incredibly difficult. While modern hearing aids provide amplification, they struggle to pinpoint exactly which speaker the user wants to hear. AAD uses Electroencephalography (EEG) to decode brain activity and identify the target speaker. The technical challenge lies in the noisy, non-linear nature of EEG data. Most existing models require long "decision windows" (several seconds) to be accurate, which is impractical for natural conversation.
Proposed Architecture: DBPNet
DBPNet addresses these challenges through a parallel processing strategy that captures complementary information from the EEG signal across two specialized pathways.
- Temporal Attentive Branch (TABNet): This branch uses Common Spatial Pattern (CSP) algorithms to clean the signal and a Transformer Encoder to capture dynamic, time-varying relationships. This allows the model to "attend" to specific moments in the audio stream that trigger stronger neural responses.
- Frequency Residual Branch (FRBNet): This branch decomposes EEG into five standard frequency bands and extracts Differential Entropy (DE) features. These are projected onto 2D topological maps and processed by 3D Residual Convolutions to capture spatial and inter-band interactions.
Features from both branches are concatenated and passed through a final classifier to determine the focus of attention.
Experimental Results
The model was evaluated on three major datasets: KUL, DTU, and the large-scale MM-AAD dataset.
Performance and Efficiency
A key insight of this work is that temporal and frequency features are highly complementary. As shown in the performance analysis, the fused DBPNet consistently outperforms its individual branches (TABNet and FRBNet) across various decision window lengths.
On the MM-AAD dataset with a 0.1-second decision window, DBPNet achieved a 20.4% relative improvement over the best baseline. Furthermore, DBPNet is roughly 91 times smaller than the MBSSFCC model, reducing the parameter count from ~84 million to just 0.91 million. This efficiency is critical for "edge" applications like hearing aids where battery life and processing power are strictly limited.
Conclusion
DBPNet demonstrates that fusing temporal and frequency domain features allows for high-accuracy AAD even with extremely short data windows. By significantly reducing model complexity while improving performance, DBPNet provides a viable path toward real-time, brain-steered hearing aids. Future work will investigate the use of Graph Neural Networks (GNNs) to better represent the non-Euclidean relationships between EEG electrode locations.
Summary updated