TL;DR
This paper introduces sDREAMER, a novel transformer-based model for automatic sleep staging that effectively integrates information from multiple physiological signals (EEG and EMG). It achieves state-of-the-art performance by introducing a "mixture-of-modality-experts" architecture and a self-distillation training strategy, enabling it to handle both single and multi-channel inputs robustly.
What Problem Does This Paper Address?
Accurately classifying sleep stages is crucial for understanding sleep disorders and overall health. Traditional methods rely on expert manual annotation of polysomnography (PSG) data, which is time-consuming and subjective. While deep learning has shown promise, existing automated methods often struggle with two key issues:
- Limited Cross-Modality Interaction: Many models treat different physiological signals (like EEG and EMG) in isolation or fuse them late in the process, failing to capture the rich interdependencies between them. This is problematic because these signals are inherently noisy, and cross-referencing can significantly improve robustness.
- Lack of Unified Input Handling: Current models are typically trained for either single-channel or multi-channel inputs, but not both. This means a model trained on EEG and EMG cannot be directly used for inference on just EEG data, limiting its practical applicability.
The authors aim to address these limitations by developing a unified model that can leverage multi-modality information for improved performance and also generalize to single-modality inputs.
Methods
The core of sDREAMER is the Mixture-of-Modality-Experts (MoME) transformer. This architecture features:
- Three Pathways: Separate pathways for EEG, EMG, and a "mixed" signal, allowing for specialized processing of each modality and their combined interactions.
- Partially Shared Weights: While attention mechanisms are shared across pathways, feed-forward networks (FFNs) are modality-specific ("experts"). This design encourages alignment across modalities while retaining modality-specific feature extraction.
- Self-Distillation: A novel training scheme where the "mixed" pathway acts as a teacher to guide the EEG and EMG pathways. This process uses KL-divergence to distill knowledge from the mixed pathway's predictions to the single-modality pathways, further enhancing cross-modality information flow and improving mono-modal performance.
- Unified Input Handling: The model is trained with multi-channel inputs but can perform inference on single-channel inputs by selecting the appropriate pathway. This is achieved through a flexible architecture that doesn't require retraining for different input configurations.
- Hierarchical Structure (Sequence SDREAMER): For sequence-level staging, the model employs an epoch encoder to capture within-epoch context and a sequence encoder to capture inter-epoch dependencies, building a hierarchical representation.
Datasets
The study utilizes a publicly available mice sleep dataset. This dataset consists of EEG and EMG signals collected from C57BL/6 mice, with expert-annotated sleep stages (Wake, SWS, REM) at a one-second resolution. The dataset comprises records from 16 mice, with 12 used for training and 4 for testing, ensuring a subject-wise split for robust evaluation.
Key Insights and Ideas
- MoME Architecture's Power: The MoME architecture, with its modality-specific experts and shared attention, proves highly effective for sleep staging by enabling richer cross-modal interactions than traditional fusion methods.
- Self-Distillation for Robustness: Using the mixed pathway as a teacher for single-modality pathways is a clever way to inject cross-modal supervision, significantly boosting performance, especially for single-modality inference.
- Unified Model Advantage: The ability to train once and infer on various input configurations (single vs. multi-channel) is a significant practical advantage, reducing the need for separate models and improving generalization.
- Implicit Alignment: The t-SNE visualizations suggest that the MoME architecture and self-distillation implicitly lead to aligned latent spaces across different pathways, indicating effective cross-modal learning without explicit alignment supervision.
Key Results
- State-of-the-Art Performance: sDREAMER (both epoch and sequence versions) outperforms existing transformer-based sleep scoring methods on multi-channel input by significant margins.
- Superior Single-Channel Inference: For single-channel inference, sDREAMER achieves better results than transformer-based models trained solely on single-channel signals, demonstrating its generalization capabilities.
- Quantitative Gains: For example, the sequence SDREAMER achieves an accuracy of 91.72% and an F1-score of 87.64% on multi-modal input, outperforming the next best method (SeqCross-Attention Transformer) by over 0.79% in accuracy.
- Ablation Study Success: Ablation studies confirm the importance of self-distillation and joint attribute encoding (positional and modality embeddings) for achieving optimal performance.
Limitations and Future Work
- Expert Annotation Dependency: The model's performance is currently evaluated against a single expert's annotation. The authors suggest that using datasets annotated by multiple experts could provide a more robust evaluation and potentially reveal further improvements.
- Potential for Human Replacement: They propose future work to investigate whether the model can outperform newly trained students, hinting at its potential to replace human scorers in the future.
- Dataset Size and Modalities: While the mouse dataset is valuable, future work could explore larger, more diverse human sleep datasets and incorporate additional physiological signals beyond EEG and EMG.
Summary updated