SPINDLE: End-to-end learning from EEG/EMG to extrapolate animal sleep scoring across experimental settings, labs and species

Ðorðe Miladinovic, Christine Muheim, Stefan Bauer, Andrea Spinnler, Daniela Noain, Mojtaba Bandarabadi, Benjamin Gallusser, Gabriel Krummenacher, Christian R. Baumann, Antoine Adamantidis, Steven A. Brown, Joachim M. BuhmannPLOS Computational Biology 2019 · DOI · OpenReview

Introduction

Sleep research relies heavily on the accurate classification of sleep states—Wake, Non-Rapid Eye Movement (NREM), and Rapid Eye Movement (REM)—from electroencephalogram (EEG) and electromyogram (EMG) recordings. Traditionally, this classification, known as sleep scoring, is performed manually by human experts. However, manual scoring is a labor-intensive, time-consuming, and subjective process, often leading to significant inter-rater variability. While automated solutions have been proposed, they frequently struggle with "domain shift"—the performance drop observed when a model trained on data from one laboratory or species is applied to another.

The Challenge of Domain Shift

The primary obstacle to universal automated sleep scoring is the variability in recording environments. Differences in electrode placement, signal-to-noise ratios, sampling rates, and even the genetic strains of the animals (e.g., mice vs. rats) create a "domain gap." Most existing machine learning models require extensive re-training or manual calibration to adapt to these differences, which limits their utility in large-scale, multi-center studies.

The SPINDLE Framework

To address these challenges, researchers developed SPINDLE (Sleep Phase Identification with Neural networks for Domain-invariant LEearning). SPINDLE is a deep learning framework designed to provide human-level accuracy while maintaining robustness across diverse experimental settings.

Hybrid Architecture

The framework utilizes a two-stage hybrid architecture:

  1. Convolutional Neural Networks (CNNs): Unlike traditional methods that rely on pre-defined spectral features, SPINDLE employs an end-to-end CNN to process raw spectrograms. This allows the system to automatically learn the most discriminative features for sleep state classification. The framework uses two distinct CNNs: one dedicated to identifying signal artifacts (noise) and another to predict the probabilities of each sleep state.
  2. Hidden Markov Model (HMM): To ensure physiological plausibility, the raw predictions from the CNN are passed through an HMM. The HMM uses Viterbi decoding to filter out impossible or highly unlikely state transitions (such as a direct jump from Wake to REM in mice), effectively acting as a biological constraint on the model's output.

Preprocessing for Generalization

To minimize the impact of magnitude differences across recording setups, SPINDLE incorporates specific preprocessing steps, including log-transformation and z-score normalization of the signals. This standardization is key to its ability to generalize across different labs and species.

Experimental Validation

The robustness of SPINDLE was tested using a diverse dataset comprising 22 animals (14 mice and 8 rats) across four distinct cohorts. These cohorts included wildtype animals as well as genetic mutants, allowing the researchers to verify if the model could detect pathological changes in sleep patterns. Data was sourced from three independent laboratories (BrownLab, BaumannLab, and TidisLab), each with its own surgical procedures and recording parameters. To establish a "gold standard," the data was double-scored by five human experts.

Key Findings

SPINDLE demonstrated remarkable performance across all test conditions:

  • Human-Level Accuracy: The model achieved agreement rates of 93% to 99% with human experts, matching the level of consensus typically found between two human scorers.
  • Exceptional Generalization: Remarkably, a model trained on only two wildtype mice from a single lab was able to accurately score data from different species (rats) and different experimental conditions without any additional fine-tuning.
  • Superior REM Detection: SPINDLE significantly outperformed existing state-of-the-art tools, particularly in identifying REM sleep, which is often the most difficult state to classify automatically.
  • Phenotypic Validation: The system successfully identified the same mutation-induced sleep alterations in genetic models that were previously identified by human experts, proving its utility for actual biological research.

Conclusion

SPINDLE represents a significant advancement in automated sleep research. By combining the feature-learning capabilities of CNNs with the physiological constraints of HMMs, it overcomes the long-standing issue of domain shift. Its ability to provide fast, reliable, and generalizable sleep scoring—processing 24 hours of data in under two minutes—makes it a powerful tool for high-throughput sleep studies across the scientific community. The framework is currently available as a web service to facilitate its adoption by researchers worldwide.

Summary updated

BibTeX
@article{miladinovic2019spindle,
  author = {Ðorðe Miladinovic and Christine Muheim and Stefan Bauer and Andrea Spinnler and Daniela Noain and Mojtaba Bandarabadi and Benjamin Gallusser and Gabriel Krummenacher and Christian R. Baumann and Antoine Adamantidis and Steven A. Brown and Joachim M. Buhmann},
  title = {{SPINDLE: End-to-end learning from EEG/EMG to extrapolate animal sleep scoring across experimental settings, labs and species}},
  journal = {PLOS Computational Biology},
  year = {2019},
  doi = {10.1371/journal.pcbi.1006968},
}
Figure