Supervising Sleep Stagers on New Clinical Populations with Their Sleep Reports

Automatic sleep staging labels each 30-second window of an overnight recording as Wake, N1, N2, N3 or REM. These models lose accuracy when moved to a new clinic or patient population, and the usual fix, having a sleep technologist relabel hundreds of windows per night, is exactly the expert bottleneck that makes the fix unaffordable. A clinical sleep study almost always produces something much cheaper, though: a one-page report. The report gives total sleep time, the percentage of the night spent in each stage, how long the patient took to fall asleep and to reach first REM, and how many times they woke up. In archived records and in large cohort studies, that report is often the only thing kept or shared.

We propose to use those reports directly as training signal for adapting a sleep stager to a new population. Each reported number is a known function of the unknown sequence of stages. Placing a small sequence model over the network's per-window outputs lets us compute the expected value of every such number under the model and penalize disagreement with the report. Stage percentages alone say only how much of each stage occurred. Latencies and awakening counts also constrain where the stages fall in the night and how often they switch. The second property is what distinguishes this from simply matching proportions.

Core Claim

Patient-level summary statistics that are functions of the stage sequence can replace dense epoch labels for adapting sleep staging to an unseen clinical population. They beat both unsupervised alignment and proportion-only weak supervision, and the gain concentrates in ambiguous transition stages such as N1. The idea behind the claim is that stage percentages are unchanged when the epochs of a night are shuffled, so they cannot tell the model where a stage occurs. Latency and transition-count constraints do depend on order. They rule out most shuffled solutions and should therefore carry most of the extra information.

What Exists and What Is Missing

The closest precedent is Domain Adaptation for Ulcerative Colitis Severity Estimation Using Patient-Level Diagnoses. It adapts an endoscopy severity classifier to a new hospital using only each patient's recorded maximum severity, enforced by a loss that forbids any frame from exceeding that maximum. It lifts accuracy from 52.1% under unsupervised adaptation to 71.4%. That paper motivates our proposal because it shows routine patient-level records can beat both unsupervised adaptation and 5% target labeling. It also marks our departure. Its constraint relies on an ordinal scale and treats frames as an unordered bag. Sleep stages are categorical and their order matters, so neither assumption carries over. Our constraints are expectations of order-dependent functionals of a labeled sequence, not an upper bound on an ordinal score.

The sleep-generalization work that supplies our backbones and our main competitors assumes dense labels throughout. U-Sleep trains a fully convolutional network on about 20,000 densely scored recordings from 16 studies and reaches F1 0.76 on the apnea cohort DOD-O. It gets around the bottleneck by pooling data rather than by using labels more efficiently. SleepDG aligns feature means and covariances across source datasets, plus correlations over 20-epoch windows, and generalizes to an unseen dataset without any target labels. MSTGCN learns subject-invariant EEG features with graph convolutions and a gradient-reversal layer. SleepSMC transfers knowledge across sensor types under dense labels.

None of these uses any target-population information. That is a strength for zero-shot deployment, but it wastes the reports that clinics already hold. Two further papers constrain the design. Visualising convolutional neural network decisions in automated sleep scoring and A computationally efficient algorithm for wearable sleep staging in clinical populations document that N1 and transition boundaries are ambiguous even for human raters, especially in fragmented clinical sleep. Reports therefore carry rater noise, and the loss must tolerate it rather than treat reports as exact.

Research Program

The model has three parts. A per-epoch encoder, initialized from a source-trained stager such as a U-Sleep or SleepDG backbone, produces emission scores. On top sits a linear-chain conditional random field, a probabilistic sequence layer that scores whole stage sequences using per-epoch scores plus learned stage-to-stage transition weights. Forward-backward inference gives exact expectations of any additive function of the sequence.

Most report fields are additive. Stage percentages sum indicators, and awakening counts sum Wake-after-sleep transitions. The order-dependent fields become additive once we enlarge the state with a small flag. For example, REM latency is the number of sleep epochs before the first REM epoch, which becomes additive once each state also records whether REM has occurred yet. This at most quadruples the state space, keeping inference linear in night length. That matters because SleepDG's correlation alignment is quadratic in length and so limited to 20-epoch windows; our constraints act on the whole night.

For a target night $x$ with reported statistics $r_k$, $k=1,\dots,K$, the adaptation loss is

$$ \mathcal{L}(x) = \sum_{k=1}^{K} w_k \, \max\!\left(0,\; \big|\mathbb{E}_{p_\theta(y \mid x)}[f_k(y)] - r_k\big| - \epsilon_k\right)^2 + \lambda\, \mathcal{L}_{\text{src}}. $$

Here $y$ is the stage sequence and $f_k$ is the function computing the $k$-th report field from it. $p_\theta$ is the conditional random field's distribution over sequences. $\epsilon_k$ is a tolerance set from rounding and from inter-rater disagreement on that field. $w_k$ normalizes fields to comparable scales. $\mathcal{L}_{\text{src}}$ is the ordinary supervised loss on labeled source nights, weighted by $\lambda$ to prevent drift.

Matching expectations can be satisfied by a spread-out, uncertain posterior. We therefore add a small entropy penalty and study whether it is needed, since the self-training literature, for example Learning to Segment Medical Images with Scribble-Supervision Alone, warns that confident wrong targets propagate.

We build the report fields that a standard clinical report contains, apply them in simulation and on real archived summaries, and analyze which fields carry information. The simulation computes report fields from the dense labels of benchmark datasets, rounds them as reports do, and then discards the dense labels for training. This lets us measure accuracy against hidden ground truth. It is a simulation of the regime, not the regime itself. We will say so, and we will also test the approach with fields perturbed by the empirical disagreement between scorers.

Baselines

The naive reference is the source-trained stager applied to the target population with no adaptation, using the same backbone as every other arm. It measures the raw gap.

The strongest existing unsupervised competitor is SleepDG, run with its own leave-one-dataset-out protocol over SleepEDFx, HMC, ISRUC, SHHS and P2018. It reports an average cross-domain accuracy of 75.03% and macro-F1 of 69.64%, against 68.76% accuracy for its unaligned baseline (Generalizable Sleep Staging via Multi-Level Domain Alignment). We will reproduce it with the same encoder, epochs and tuning budget as our method before comparing. We will also combine it with our loss, since the two are complementary. Beating SleepDG means a higher average macro-F1 under the identical protocol, with the difference exceeding seed-to-seed variation across at least five seeds.

The strongest large-scale competitor is U-Sleep used zero-shot. It reports F1 0.76 on DOD-O and 0.79 on DOD-H (U-Sleep). For DOD-O we will compare against that published figure and against a U-Sleep initialization adapted with our loss. Beating it means exceeding 0.76 F1 on DOD-O. We will only claim this if DOD-O is confirmed not to be in the pretraining pool of the checkpoint we use.

The key ablation is proportion-only weak supervision. This is the standard "learning from label proportions" setup with the same conditional random field and loss, restricted to stage percentages. It isolates the claim that order-dependent fields matter. A second ablation keeps the conditional random field but drops the report loss, separating the effect of the structured output layer from that of the supervision. For reference, the Yamaguchi et al. max-severity approach does not apply directly to non-ordinal stages. Its reported 71.4% on LIMUC-to-private is from a different task, and we cite it only as motivation.

The upper bound is fully supervised fine-tuning on the dense target labels. The low-budget semi-supervised arm densely labels 5% of target nights. We choose whole nights rather than scattered epochs, because that is how a clinic would actually commission scoring.

Why not just…?

Why not just match the stage percentages, which is the obvious weak-label idea? This is the real threat. If proportions alone recover most of the gain, the extra structure is not worth its complexity. Shuffled solutions satisfy proportion constraints exactly, so we expect proportion-only training to fix class balance, raising N3 and REM recall in populations where these shrink, while leaving transition-heavy N1 and Wake-after-sleep errors largely untouched. The ablation settles this directly. If adding latencies and counts does not improve N1 F1 and the number of stage switches over proportions alone, we will report that the simpler method suffices.

Why not just use unsupervised alignment such as SleepDG, which needs no target information at all? Alignment matches feature statistics across sites. In a clinical population, however, the true stage distribution itself shifts, for example less N3 and more awakenings in apnea. Pure feature alignment can then pull target features toward source class proportions that are wrong for the target. Reports carry exactly the shifted proportions. The head-to-head test and the combined arm show whether the information is complementary. If SleepDG plus our loss does not beat SleepDG alone, the reports add nothing.

Why not just pool more densely labeled data, as U-Sleep does? On populations that resemble the pooled studies, this may simply win, and DOD-O suggests it often does. The case for our method is populations that are poorly represented, where no pool exists. We test this with the leave-one-dataset-out protocol and with the clinical ISRUC-S1 sleep-disorder subset. We report honestly where zero-shot U-Sleep is already at ceiling.

Why not just pay a technologist to score a few target nights? The file's evidence is indirect. Yamaguchi et al. found that patient-level records beat 5% labeled target data in endoscopy, but sleep may differ. Our 5%-of-nights arm, and a curve over how many nights are densely scored, measure the exchange rate. That rate is how many report-only nights equal one scored night.

Decisive Checks

We run the SleepDG leave-one-dataset-out protocol across SleepEDFx, HMC, ISRUC, SHHS and P2018. We additionally test on DOD-O, on ISRUC-S1 as a disorder-heavy target, and on MASS-SS3 for comparison with MSTGCN's published within-dataset figures. The reported metrics are macro-F1, Cohen's kappa, per-stage F1, and the error in predicted number of stage switches per night. The last metric directly tests whether sequence structure is recovered.

The core check is the ordering from the no-adaptation baseline to SleepDG, then to proportions only, then to full report supervision, and finally to the dense-label oracle. For the claim to hold, most of the improvement from proportions to full reports must appear in N1 and in the count of stage switches.

The robustness check perturbs report fields with rater-level noise and then drops fields one at a time. The drop-one analysis tells us which clinical fields a clinic must retain.

A failure check examines per-night posteriors for collapse. These are cases where expectations match the report but the most likely sequence is physiologically implausible, such as REM directly after Wake at sleep onset in non-narcoleptic patients.

If the full-report model does not beat proportion-only training on N1 and on the stage-switch count, the central claim is wrong and we will say so.