Distilling Patient Differences, Not Just Labels, From Sleep EEG Into Infrared Video

Hospital sleep studies record brain activity with multi-channel electroencephalography (EEG, electrodes on the scalp that measure brain oscillations). They produce two things. The first is a stage label for every 30-second window of the night. The second, which matters more clinically, is a picture of how one patient's night differs from other people's: how fragmented it is, how much light sleep it contains, how often the patient switches between stages. Recent work shows that a camera-only model can learn the first product from the EEG. Seeing your sleep stage trains a student network on non-contact infrared video to imitate an EEG teacher and raises window-level staging accuracy from 52.5% to 64.4%. Nobody has asked whether that student preserves the second product. A student that is right two-thirds of the time can still reorder patients badly, because its errors are not random. It confuses light sleep with wakefulness, and that confusion is worst in exactly the fragmented, atypical sleepers a clinician most wants to single out.

This note proposes to distill the between-patient structure directly. For each patient the teacher produces a normative deviation profile: how far that patient's EEG-derived sleep architecture sits from age-matched norms. The student is trained so that distances between patients in its video-derived embedding match distances between their teacher profiles. It is still trained on the usual window-level objective. The question is whether a camera can inherit the teacher's map of clinical heterogeneity, the fact that patients sharing a diagnosis differ biologically, rather than only its per-window labels. The test is whether this beats the obvious pipeline of predicting stages first and computing profiles from the predicted hypnogram.

Core Claim

The claim is that per-window accuracy and per-patient fidelity are different targets under cross-modal distillation. When the commodity sensor's errors are correlated within a patient, a subject-level relational objective recovers the teacher's patient geometry better than any gain in window accuracy would suggest. Correlated errors come from a quilt that occludes one whole night, or from a sleep style whose N1 phase is invisible on camera. The heterogeneity literature supplies the reason to care about geometry rather than means. Alpha Waves as a Neuromarker of Autism found no group mean difference in EEG alpha power. A LOESS normative model, which fits smooth age-dependent norms and scores each person's deviation from them, did reveal significantly higher variability in the autism group. The clinically meaningful signal lived in individual deviations, not averages. The treatment literature makes the same point. In Peripheral inflammatory biomarkers define biotypes of bipolar depression, the whole cohort showed no benefit, but a data-derived biotype did. If commodity sensors are ever to carry stratification outside the lab, they have to preserve who is unusual and in which direction.

What Exists and How This Differs

Two papers already connect specialized-to-commodity transfer with clinical settings, and both transfer labels rather than heterogeneity. The sleep paper uses Structure-Aware Contrastive Distillation, a contrastive loss with hard-negative memory banks that aligns video and EEG window embeddings. Its targets are uniform sleep stages, and it reports that its predictions correlate with the apnea-hypopnea index. This paper supplies our base method and our dataset. Democratizing Clinical Risk Prediction with Cross-Cohort Cross-Modal Knowledge Transfer distills a genetics-plus-health-records teacher into a records-only student for binary risk prediction. It reports AUROC 79.94% on All of Us and 71.82% after transfer to a local hospital, and it does not examine subtyping or subgroup stability. D-vlog shows the cost of the alternative. It trains commodity audio and face models directly on a monolithic depression label, reaches F1 63.50%, has no biological anchor, and shows a large gap between male and female subjects.

The proposal differs from all three in its training target. We transfer a patient-by-patient distance structure defined by the specialized sensor, and we evaluate whether that structure survives the move to the commodity sensor.

Two further papers constrain the design. Population heterogeneity in clinical cohorts affects the predictive accuracy of brain imaging shows that accuracy falls as demographic distance between training and test sets grows (r ≈ −0.67), and that ComBat and linear deconfounding do not fix this. Stability across strata must therefore be measured directly, not assumed after harmonization. Representation Learning for Integrating Multi-domain Outcomes shows that modeling observed measures as indicators of latent states beats aggregate sums on STAR*D. This supports treating the deviation profile, rather than a single summary score, as the object to preserve.

Research Program

Everything is built on the synchronized S³VE dataset from the sleep paper. It contains 105 subjects, over 1,100 hours of recordings, and 154,573 thirty-second clips with paired polysomnography EEG and infrared video. It is the only resource in our file with a specialized and a commodity sensor recorded together.

For each subject $i$, we compute a vector $a_i$ of sleep-architecture statistics from the EEG teacher's hypnogram (the stage-by-time sequence for the night). The statistics are sleep efficiency, the fraction of time in each stage, transition rates between stages, wake-after-sleep-onset bouts, and a fragmentation index. On training subjects only, we fit LOESS norms over age, following the alpha-wave paper. Each subject's deviation profile is

$$d_i = \frac{a_i - \mu(\text{age}_i)}{\sigma(\text{age}_i)},$$

where $\mu$ and $\sigma$ are the age-conditional mean and spread of each statistic.

The student is the SACD video network with one addition. It pools its window embeddings into a patient embedding $e_i$. It is trained with the SACD loss plus a relational term:

$$\mathcal{L} = \mathcal{L}_{\text{SACD}} + \lambda \sum_{i,j} \left( s\,\lVert e_i - e_j \rVert - \lVert d_i - d_j \rVert \right)^2 .$$

Here $\lambda$ weights the relational term and $s$ is a learned scale.

With roughly 84 training subjects, true patient pairs are scarce. We therefore also form pseudo-patients from multi-hour sub-night segments, each with its own teacher profile. This increases the number of pairs while keeping the loss at the subject level. We will also test a variant that regresses $d_i$ from $e_i$ directly. It isolates whether relational matching adds anything over plain supervision of the targets.

Analysis covers three things. The first is patient-geometry fidelity: the Spearman correlation between student and teacher distance matrices, and the agreement between k-means clusters, measured by adjusted Rand index under bootstrap. The second is stability across strata. We compare that fidelity across age tertiles, sex, and nights with and without occlusion, since occlusion is flagged as a failure mode in the source paper. The third is external validity: whether the student's clusters and deviation axes keep the teacher's association with apnea severity.

We will not claim to discover clinical biotypes. With 105 subjects, the honest goal is to show that the student reproduces whatever structure the EEG defines.

Baselines

The naive reference is a video-only student trained on stage labels without distillation. It achieves 52.5% window accuracy on S³VE according to Seeing your sleep stage. Its predicted hypnograms are converted into deviation profiles with the same LOESS norms. This measures how much patient structure a camera captures with no teacher at all.

The strongest competitor is SACD itself, which reports 64.4% window accuracy on S³VE, above CRD and CCL at 62.3% each. We use it in a predict-then-profile pipeline: the SACD student outputs hypnograms, and deviation profiles are computed from them. It is the right rival because it is the best available staging model. If better staging automatically gives better profiles, our loss is unnecessary. No reported result was found on Lacuna for subject-level profile or cluster recovery from any S³VE model, so we must measure this ourselves.

Both SACD and our model share the same backbone and the same subject-disjoint five-fold splits. Norms are fitted only within each training fold. Each method gets an identical hyperparameter search budget. We count our method as beating SACD if two conditions hold. First, the distance-matrix Spearman correlation and cluster adjusted Rand index must exceed SACD's with non-overlapping bootstrap 95% intervals. Second, window accuracy must stay within one point of 64.4%.

Two ablations isolate the key ingredient. The first removes the relational term, which recovers SACD. The second replaces normative deviations $d_i$ with raw statistics $a_i$. This tests whether the normative framing, rather than just any subject-level target, drives the gain. The low-cost clinical reference is age, sex, and apnea index alone, used to predict teacher clusters. It shows how much of the structure is already captured without any sensor.

Why not just…?

Why not just compute the profiles from the best stage predictor? This is the real threat. It will win if the student's staging errors are roughly independent across windows and subjects. The source paper reports that N1 and wakefulness remain visually ambiguous and that quilts block tracking. Both failures act on whole nights and on particular sleep styles, which is exactly the correlated-error regime where it should fail. The head-to-head comparison in Baselines settles the question. If predict-then-profile matches us, we will report that window-level distillation already suffices for stratification. That is a useful negative result for anyone planning commodity-sensor subtyping.

Why not just regress each patient's teacher profile from video? Direct regression of about ten deviation scores from about 84 subjects is likely to overfit. It also discards the within-night supervision that makes SACD work. We run it as an explicit variant: regression head only, then regression plus SACD. If it matches the relational loss, we adopt the simpler objective.

Why not just harmonize the student's outputs across demographic groups afterward with ComBat? Population heterogeneity in clinical cohorts found that ComBat and linear covariate regression failed to recover accuracy lost to demographic distance. The reason is that diversity changed multivariate feature relationships, not just their offsets. We nevertheless apply ComBat to both SACD and our outputs and report fidelity across strata with and without it. That way, any stability we claim is not something post-hoc harmonization could have delivered.

Why not just use the apnea index and questionnaires, which need no camera? Clinical scores are cheap, and Patterns of schizophrenia symptoms found that linear models on such scores matched nonlinear ones. But the apnea index is itself a lab measurement, and scalar sums lose the multidimensional structure, as Representation Learning for Integrating Multi-domain Outcomes argues. The clinical-reference baseline quantifies how much of the teacher's patient structure these scores explain. The video student must add to it to be worth deploying.

Decisive Checks

The proposal succeeds if the relational student beats predict-then-profile SACD on distance-matrix correlation and cluster agreement while holding window accuracy near 64.4%. It must also degrade less than SACD on occluded nights and in the age stratum furthest from the training norm, and it must preserve the teacher's association with apnea severity. The mechanism is supported only if a specific pattern appears. SACD's profile error should concentrate in subjects with high teacher-measured N1 fraction and fragmentation, and our loss should reduce error specifically in that group. If the gain instead spreads uniformly, or disappears once the regression variant is included, the correlated-error explanation is wrong, and we will say so.