Towards Domain Generalization for ECG and EEG Classification: Algorithms and Benchmarks

Aristotelis Ballas, Christos Diou

Abstract

Machine learning models for biosignal classification often fail when deployed in real-world clinical settings due to distribution shifts between training and testing data. This paper introduces BioDG, a comprehensive open-source benchmark and a novel neural network architecture designed for Domain Generalization (DG) in ECG and EEG signals. By extracting and concatenating features from multiple intermediate layers of a deep network, the proposed BioDG model captures domain-invariant signatures that improve performance on out-of-distribution (OOD) data. Experimental results demonstrate that BioDG significantly outperforms standard empirical risk minimization and existing DG algorithms adapted from computer vision, while maintaining computational efficiency.

Introduction

In clinical practice, the assumption that training and testing data are independent and identically distributed (i.i.d.) rarely holds. Factors such as varying equipment, electrode placement, and diverse patient demographics create "domain shifts" that degrade the performance of models trained on specific datasets. While Domain Generalization (DG) has been extensively explored in computer vision, its application to 1D biosignals like ECG and EEG has been limited. This work addresses this gap by providing a standardized benchmark for biosignal DG and a novel architecture that leverages multi-layer feature learning to identify robust, disease-specific patterns across different hospitals and populations.

Methods

The researchers propose two primary contributions: a standardized benchmark and a specialized architecture.

The BioDG Benchmark

To simulate real-world deployment, the authors established a "leave-one-domain-out" evaluation protocol. They adapted four state-of-the-art DG algorithms—CORAL, IRM, MMD, and RSC—for 1D signals. The benchmark utilizes six ECG datasets (e.g., CPSC, PTB-XL) covering 24 diagnoses and three EEG datasets (SEED-CHI, FRA, GER) for emotion recognition. This protocol ensures that models are tested on entirely unseen domains, reflecting the challenges of cross-hospital or cross-population deployment.

Domain partitioning for ECG and EEG datasets in the BioDG benchmark.

The BioDG Architecture

The core idea behind the BioDG model is that "domain-invariant" features—the essential markers of a condition—are often distributed across different levels of abstraction in a neural network. Standard models typically rely only on the final layer for classification, which may be too specialized to the training domain's "style."

BioDG attaches "concentration pipelines" to multiple intermediate layers of a ResNet backbone. Each pipeline consists of a 1x1 convolution for data compression, spatial dropout for regularization, and average pooling. The resulting features from all layers are concatenated, providing the classifier with a multi-scale view of the signal simultaneously.

The BioDG model architecture for ECG and EEG, featuring multi-layer concentration pipelines.

Results

The BioDG model consistently outperformed baseline Empirical Risk Minimization (ERM) and other DG methods across both modalities:

  • ECG Classification: BioDG achieved a Macro-F1 score of 35.88% on OOD data (using a ResNet-18 backbone), significantly higher than the 23.40% achieved by the ERM baseline. Furthermore, it successfully identified 15 cardiac conditions, whereas the baseline could only reliably detect 11.
  • EEG Classification: In cross-nationality emotion recognition, BioDG reached 68.04% accuracy when testing on the German dataset after training on Chinese and French data, surpassing the next best method (CORAL) by approximately 1%.
  • Efficiency: Despite the architectural complexity, the model adds less than 1% to the total MACs (Multiply-Accumulate operations), making it suitable for real-time monitoring. However, it does increase GPU memory usage by approximately 3.5 GB due to the multi-layer feature extraction.

Conclusion

BioDG represents a significant step toward robust biosignal AI. By moving beyond single-layer feature extraction, the model better handles the variability inherent in medical data. Future work will focus on optimizing memory usage, incorporating attention mechanisms to weight layer importance, and enhancing model explainability through saliency maps to aid clinical adoption.

Summary updated

Abstract

Despite their immense success in numerous fields, machine and deep learning systems have not yet been able to firmly establish themselves in mission-critical applications in healthcare. One of the main reasons lies in the fact that when models are presented with previously unseen, Out-of-Distribution samples, their performance deteriorates significantly. This is known as the Domain Generalization (DG) problem. Our objective in this work is to propose a benchmark for evaluating DG algorithms, in addition to introducing a novel architecture for tackling DG in biosignal classification. In this article, we describe the Domain Generalization problem for biosignals, focusing on electrocardiograms (ECG) and electroencephalograms (EEG) and propose and implement an open-source biosignal DG evaluation benchmark. Furthermore, we adapt state-of-the-art DG algorithms from computer vision to the problem of 1D biosignal classification and evaluate their effectiveness. Finally, we also introduce a novel neural network architecture that leverages multi-layer representations for improved model generalizability. By implementing the above DG setup we are able to experimentally demonstrate the presence of the DG problem in ECG and EEG datasets. In addition, our proposed model demonstrates improved effectiveness compared to the baseline algorithms, exceeding the state-of-the-art in both datasets. Recognizing the significance of the distribution shift present in biosignal datasets, the presented benchmark aims at urging further research into the field of biomedical DG by simplifying the evaluation process of proposed algorithms. To our knowledge, this is the first attempt at developing an open-source framework for evaluating ECG and EEG DG algorithms.
BibTeX
@article{ballas2023towards,
  author = {Aristotelis Ballas and Christos Diou},
  title = {{Towards Domain Generalization for ECG and EEG Classification: Algorithms and Benchmarks}},
  journal = {IEEE Trans. Emerg. Top. Comput. Intell},
  year = {2023},
}
Figure