ATTA: Adaptive Test-Time Adaptation for Multi-Modal Sleep Stage Classification

Ziyu Jia, Xihao Yang, Chenyang Zhou, Haoyang Deng, Tianzi Jiang

Abstract

Sleep stage classification is critical for diagnosing sleep disorders, yet deep learning models often struggle with inter-subject variability and domain shifts. We present ATTA (Adaptive Test-Time Adaptation), a novel framework designed to improve multi-modal sleep stage classification by adapting to new subjects in real-time. ATTA addresses three primary challenges: catastrophic forgetting, static modality weighting, and rigid learning rates. Our framework incorporates an Intra-modal Retained-Adaptive Module to balance general and subject-specific knowledge, an Inter-modal Contribution Assessment Module to dynamically weight sensor reliability, and an Adaptive Learning Rate Strategy based on a feature memory bank. Evaluated on the SleepEDF and SHHS datasets, ATTA significantly outperforms state-of-the-art test-time adaptation methods, achieving up to 78.04% accuracy in cross-dataset scenarios.

Introduction

Accurate sleep stage classification—categorizing sleep into REM, light, and deep stages—is fundamental to sleep medicine. While deep learning has advanced this field, models often suffer from performance degradation when applied to new subjects due to inter-subject variability. This "domain shift" occurs because physiological signals like EEG and EOG vary significantly between individuals, making a static model trained on one population less effective for another.

Existing Test-Time Adaptation (TTA) methods attempt to update models on-the-fly as they encounter new data. However, they face three major hurdles: catastrophic forgetting of general patterns during rapid adaptation, static modality weighting that ignores sensor-specific reliability for different patients, and rigid learning rates that fail to account for how much a new patient's data differs from the training set.

The ATTA Framework

To address these challenges, we propose the Adaptive Test-Time Adaptation (ATTA) framework. ATTA is designed to provide stable yet flexible adaptation for multi-modal sleep data by balancing previously learned sleep patterns with new, subject-specific data.

The ATTA architecture features dual-stream processing for each modality and a dynamic contribution assessment module.

The framework consists of three core components:

  1. Intra-modal Retained-Adaptive Module: For each sensor (EEG and EOG), ATTA maintains two models in parallel. A "Retained" model uses slow momentum-based updates to preserve foundational sleep knowledge from the source data. Simultaneously, an "Adaptive" model updates rapidly to capture subject-specific nuances. By combining their outputs, the system stays grounded while remaining flexible.
  2. Inter-modal Contribution Assessment Module: This module dynamically determines the reliability of each sensor. By measuring the divergence (using KL divergence) between the Retained and Adaptive models for a specific modality, the system can downweight sensors that show unstable or unrecognizable patterns for a given patient in a specific 30-second window.
  3. Adaptive Learning Rate Strategy: Utilizing a "memory bank" of previously encountered features, the model adjusts its learning rate based on the similarity of the current sample to historical data. This ensures aggressive adaptation for "outlier" patients and stable maintenance for those who match standard training patterns.

Experimental Results

We evaluated ATTA using two large-scale public datasets: SleepEDF (153 recordings from 78 subjects) and SHHS (329 individuals). The primary test involved "mutual migration," where a model trained on one dataset is adapted and tested on the other to simulate real-world hardware and population shifts.

ATTA significantly outperforms existing TTA and domain generalization methods across both cross-dataset evaluation scenarios.

As shown in the results table, ATTA achieved a state-of-the-art accuracy of 78.04% on the SleepEDF → SHHS task, representing a massive jump over the "No Adapt" baseline (64.09%) and existing methods like SAR (72.75%). Similar gains were observed in the SHHS → SleepEDF direction, where ATTA reached 74.15% accuracy.

Ablation studies further confirmed the importance of the dual-model structure. Removing the "retained" component led to a sharp drop in accuracy (from 78.04% to 71.32%), proving that preserving source knowledge is just as critical as the adaptation itself to prevent the model from spiraling into instability when encountering noisy test data.

Conclusion

ATTA provides a robust solution for real-time adaptation in clinical sleep monitoring. By balancing stability with flexibility and dynamically assessing sensor reliability, it overcomes the traditional pitfalls of test-time adaptation. While the dual-stream architecture introduces some computational overhead, its superior accuracy and robustness to inter-subject variability mark a significant step forward for personalized sleep medicine.

Summary updated

BibTeX
@inproceedings{jia2024atta,
  author = {Ziyu Jia and Xihao Yang and Chenyang Zhou and Haoyang Deng and Tianzi Jiang},
  title = {{ATTA: Adaptive Test-Time Adaptation for Multi-Modal Sleep Stage Classification}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2024},
}
Figure