Where Oversampling Gains Come From in Image-Encoded Passive EEG

Passive brain-computer interfaces read a person's ongoing brain activity, such as electroencephalography (EEG), to infer states like drowsiness or overload without asking the person to do anything. The states that matter most for safety, such as a driver's microsleep or an operator's attention lapse, are rare in a long recording. Two common engineering moves follow. The first is to oversample the rare class with SMOTE, the Synthetic Minority Over-sampling Technique, which creates fake minority examples by drawing points on the straight line between a real example and one of its nearest neighbours. The second is to turn each EEG window into an image with a Gramian Angular Field (GAF). A GAF maps each sample to an angle and fills an $N \times N$ matrix with the cosine or sine of every pair of angles, so that an ordinary 2D image classifier can be used.

This note argues that the gains this pipeline reports can come from two artifacts of the pipeline rather than from better detection of the brain state. The first artifact is leakage. SMOTE's nearest neighbours in a sliding-window EEG dataset are mostly overlapping windows from the same moment. The GAFs of two overlapping windows share an exact sub-block, so synthetic minority images can carry near-copies of evaluation windows into training. The second artifact is a shortcut. A true GAF is a matrix of rank at most two, so SMOTE applied directly to GAF images produces images that no EEG signal could generate. A classifier can then learn "off-manifold texture means minority" and fail on real rare events. We propose to measure both effects on public vigilance and attention EEG and to separate them with controlled ablations. We also propose an oversampler that keeps every synthetic image a valid GAF and draws neighbours only from windows outside a time-exclusion radius. The aim is to establish which reported gains survive.

Core Claim

The claim rests on two exact facts about GAF that, as far as the compiled literature shows, nobody has connected to oversampling. Write a window rescaled to $[-1,1]$ as a vector $x$, and let $s$ be the vector with entries $\sqrt{1-x_i^2}$. The summation field is then

$$I_{\text{GASF}} = x x^\top - s s^\top ,$$

which has rank at most two. The matrix SMOTE produces in image space, $I_a + \lambda (I_b - I_a)$ with $\lambda$ drawn uniformly from $(0,1)$, generally has rank up to four. It therefore lies off the set of GAF images that any signal can produce, and its distance to that set can be measured as the spectral energy beyond the top two singular values.

The second fact concerns overlap. If two windows overlap by $N-\delta$ samples, where $\delta$ is the hop between them, and both are scaled with the same session-level constants, their GAFs share an identical $(N-\delta)\times(N-\delta)$ block, shifted along the diagonal. A convolutional network, which responds to the same pattern wherever it appears, can match that block. Under per-window scaling the shared block is only approximately preserved.

These facts give two testable predictions. The first prediction is about splits. When training and evaluation windows are not separated in time, the extra score that oversampling buys should grow with window overlap, and it should grow faster for GAF with a 2D network than for raw 1D input. The second prediction is about image-space SMOTE. It should produce synthetic images that a simple detector separates from real ones almost perfectly. A minority classifier trained on such images should rely partly on the "synthetic-ness" direction, which would show up as worse recall on real rare events than oversampling methods that stay on the set of valid GAF images.

Relation to Prior Work

The obvious combination, GAF encoding of passive EEG plus SMOTE, is not itself new in spirit. Deep Neural Networks on EEG Signals to Predict Auditory Attention Score Using Gramian Angular Difference Field stacks per-channel difference fields and trains a 2D CNN on the Phyaat dataset. It treats the task as regression, uses fixed validation splits, and never addresses imbalance or leakage. We reuse its encoding but evaluate rare-state detection under guarded splits. CerebraGloss feeds waveform images to a vision-language model for clinical EEG. It shows that the image route is gaining traction and that static splits remain the norm, but it plays no mechanistic role here.

On the oversampling side, three papers supply methods and constraints. T-SMOTE deliberately interpolates between temporally adjacent windows preceding an event. That design is sensible for early warning, but it is exactly the regime in which overlap leakage would be strongest if splits are not blocked in time, so we test it as a likely worst case rather than assume it is safe. Simplicial SMOTE samples inside higher-dimensional simplices of the neighbour graph to cover the minority region better. In image space it would push synthetic GAFs even further off the rank-two set, since a convex combination of $p+1$ rank-two matrices can reach rank $2p+2$. Concentration and excess risk bounds for imbalanced classification with synthetic oversampling proves that SMOTE's concentration error grows like $(k/n_1)^{1/d}$, where $k$ is the neighbour count, $n_1$ the number of minority examples, and $d$ the feature dimension. This constrains the design: interpolating flattened $N\times N$ images is the worst possible regime, which is one reason to interpolate in the length-$N$ signal or angle space instead. From SMOTE to Mixup for Deep Imbalanced Classification finds that naive SMOTE helps deep image models little and proposes Margin-Aware Mixup (MAMix). It motivates our question of whether the failure is worse on GAF images, where the off-manifold problem is exact rather than a matter of degree.

SMOTE and Mirrors supplies the attack machinery. Its DistinSMOTE attack finds synthetic points through collinear triplets and reaches 1.00 precision and 1.00 recall on eight UCI tabular datasets. We adapt that test to ask a different question: how much of the evaluation set is effectively reproduced inside the synthetic training set, which is a privacy question turned into an evaluation-validity question.

The contamination framing comes from language-model evaluation. DyVal shows that models strong on static benchmarks can score near zero on freshly generated tasks. We cannot generate new EEG, so our analogue is to re-split the same recordings with guards and measure how much of the score disappears. On the BCI side, The Berlin Brain–Computer Interface reports about a 10% accuracy drop when moving to subject-independent classifiers. Capsule Attention for Multimodal EEG-EOG Representation Learning reports RMSE rising from 0.0295 within participants to 0.1089 across participants on SEED-VIG. Both show how large split-dependent gaps are in this field, and that magnitude motivates measuring how much of such a gap oversampling creates.

Research Program

The study has three parts.

The first part is a split-sensitivity audit. We cut EEG recordings into windows at several overlap levels and represent each window either as raw multichannel signal or as stacked summation and difference fields. We then apply each oversampler in each of several placements: before splitting, inside the training fold with random window splits, inside the training fold with time-blocked splits separated by a guard gap, and inside leave-one-subject-out folds. The quantity of interest is the leakage gap, defined as the difference in minority detection score between the unguarded and the guarded protocol for the same model and oversampler. We model this gap as a function of overlap, representation, and oversampler.

The second part is a manifold and shortcut analysis. For every synthetic image we compute its rank-excess energy. We train a small detector to separate synthetic from real minority images and report its AUC. We then probe the minority classifier itself in two ways: we test whether its minority logit correlates with rank-excess, and we measure its recall on real evaluation minority windows against its recall on held-out synthetic windows.

The third part is a constructive fix, which we call guarded on-manifold oversampling. Neighbours are searched only among same-class windows from the same subject whose start times differ by more than the window length, so no shared block exists. Interpolation happens in the angle space $\phi = \arccos(x)$, or in the signal space, and the result is re-encoded as a GAF, so every synthetic image is exactly rank two by construction. Because the number of candidate neighbours shrinks, the exclusion radius has a cost; the bound above predicts that the cost rises as $n_1$ falls, and we will measure it.

Baselines

The naive reference point is the image classifier trained on the imbalanced data with no correction and a decision threshold tuned on training-fold validation only. Every other method must beat it under the guarded protocol to count as helping.

The strongest cheap competitor is class-weighted cross-entropy, together with focal loss, on the same GAF classifier. It changes no data and cannot leak, so it is the reference that any oversampling result must beat on real minority recall. It is run with the same architecture, epochs, and hyperparameter search budget as the oversampling arms.

The strongest interpolation-based method in the file is MAMix with deferred reweighting from From SMOTE to Mixup. The file reports minority-class accuracies of 0.79 and 0.82 on CIFAR-10 at imbalance ratio 100, against 0.63 and 0.66 for LDAM-DRW. It is the strongest because it is designed for deep image models, so we run it on GAF images with its published margin parameter tuned under the same budget.

The strongest time-series oversampler is T-SMOTE. The file reports AUPRC 10.54% above the next best competitor at a leading time of 10 timestamps on UCR/UCI data. We run it on raw windows, both with and without the time-exclusion radius.

For the representation question, the reference on SEED-VIG is LSTM-CapsAtt. The file reports cross-participant RMSE 0.1089 ± 0.0696 and PCC 0.8823 for that model, but those numbers are for continuous regression. For the thresholded rare-drowsiness detection task we pose, no reported result was found on Lacuna for SEED-VIG, so we will reimplement a 1D recurrent/convolutional baseline on raw EEG and EOG. On Phyaat, the GADF 2D CNN reports validation MAE 29.43, against 29.65 for the SVM; for a rare low-attention detection task, no reported result was found on Lacuna.

Two ablations isolate the key ingredients. The first keeps the time-exclusion radius but interpolates in image space; it isolates the manifold effect. The second interpolates on the manifold but drops the radius; it isolates the overlap effect.

Beating the field means two things. Guarded on-manifold oversampling should exceed class-weighted loss on AUPRC under leave-one-subject-out evaluation, with confidence intervals from per-subject bootstrap excluding zero. It should also show a leakage gap near zero where plain SMOTE shows a positive one.

Why not just…?

Why not just apply SMOTE inside the training fold with leave-one-subject-out splits, the standard best practice? That protocol removes the cross-subject channel. It does nothing about the image-space shortcut, which operates entirely within training data, and it is not what many within-subject passive-BCI results use; the SEED-VIG intra-participant figure is one such result. This alternative is a real threat to the leakage half of the claim. If the leave-one-subject-out leakage gap is already zero, the audit simply confirms existing practice and quantifies how much within-subject random-split numbers inflate. The shortcut analysis then carries the contribution. The split-sensitivity experiment settles this directly.

Why not just reweight the loss instead of oversampling? It may well suffice. The bound in Concentration and excess risk bounds argues that reweighting cannot fill a sparse minority support, but that argument is about plug-in classifiers, not CNNs. If class-weighted loss matches guarded oversampling, the honest conclusion is that oversampling GAF images is unnecessary, and we will report it that way.

Why not just skip GAF and run a 1D network on raw EEG? The file's evidence for GAF is thin: the Phyaat gain is 29.43 against 29.65 MAE. Both representations are run under every protocol. If 1D models match GAF once leakage is removed, that is itself a finding: part of GAF's apparent advantage would have depended on its sensitivity to shared overlap blocks.

Why not just use Mixup? Mixup in image space has the same rank-inflation problem as SMOTE, so it is included in the manifold ablation. A finding that MAMix's gains survive despite off-manifold samples would weaken the shortcut hypothesis.

Proposed Experiments

The primary dataset is SEED-VIG: 23 participants, 17-channel EEG plus 4-channel EOG over 120 minutes, with PERCLOS ground truth. We define the rare drowsy class as windows above a high PERCLOS percentile chosen on training subjects only, which gives imbalance ratios of roughly 10 to 50. The second dataset is Phyaat (25 subjects), where we threshold the lowest attention scores. As a non-EEG control, we use T-SMOTE's UCR/UCI imbalanced sets, where overlap can be varied synthetically. We vary window overlap from 0 to 90% and use five seeds. Metrics are AUPRC, balanced accuracy, recall at a fixed false-alarm rate, the leakage gap, the synthetic-versus-real detector AUC, and the rank-excess energy of synthetic images.

Three checks decide the outcome. The first check: the leakage gap for plain SMOTE on GAF should rise with overlap and exceed the gap for raw 1D input. If it is flat, the overlap mechanism is wrong. The second check: image-space SMOTE should give a synthetic-detector AUC near 1 and lower real-minority recall than on-manifold interpolation at matched synthetic counts. If it does not, the shortcut mechanism is wrong. The third check: guarded on-manifold oversampling should beat class-weighted loss under leave-one-subject-out evaluation. If it does not, the practical recommendation becomes "reweight, do not oversample." Any of these outcomes is publishable because each settles a question the current literature leaves open.