Few-Source Physical Autoencoders for Single-Trial EEG

Scalp EEG measures brain activity through several centimeters of tissue and bone, so every electrode records a blurred mixture of many brain regions plus muscle, eye, and sensor noise. A common idea for cleaning such data is the physics-informed autoencoder: a network compresses the recording into a small code, but the part that turns the code back into data is not learned. Instead it is a fixed physical simulator, so the only way to reconstruct the input is to find physically meaningful parameters. For EEG the natural simulator is the lead field, the known linear map from electrical currents on the cortex to voltages at the electrodes. We argue that plugging the lead field in as a decoder does not remove noise by itself. With thousands of cortical locations and only 64 electrodes, the lead field can produce almost any scalp pattern, including muscle noise, so the encoder can still fit noise perfectly. The physics only acts as a filter when the code is small. We therefore propose an autoencoder whose code is a handful of cortical patches, each with a compact time course, together with a per-subject head-shape code and explicit eye and muscle nuisance terms. It is trained to predict held-out electrodes through the fixed lead field.

The two-sentence version is this. Brain signal spreads to many electrodes in a way the lead field predicts, while much sensor noise is either confined to single electrodes or has scalp patterns no small set of cortical patches can produce. An encoder forced to explain masked electrodes through a few lead-field sources should therefore keep the brain signal and drop the noise. If that holds on single trials, which is where current EEG decoders and source-imaging networks give up, it gives a denoiser that also has physical meaning and could be placed in front of any existing decoder.

Core Claim

A fixed physical decoder removes noise only in proportion to how little of the sensor space its code can reach. We will make that reach small in three ways: a few sources, a restricted family of time courses, and loss computed only on masked electrodes. The claim is that doing so gives single-trial denoising that measurably beats an unconstrained low-rank autoencoder of the same size. That comparison is the ablation the whole proposal rests on.

The generative model for a trial is

$$\hat{x}_t = \sum_{k=1}^{K} L_\theta(r_k)\, s_k(t) + B\, a_t ,$$

where $\hat{x}_t$ is the predicted voltage on all electrodes at time $t$. $K$ is a small number of cortical patches (roughly 3 to 8). $r_k$ is patch $k$'s location and spatial extent on a cortical mesh. $L_\theta(r_k)$ is the scalp pattern that patch produces under head-model code $\theta$. $s_k(t)$ is the patch's time course, drawn from a low-dimensional family, namely a sum of a few Gabor-like transient atoms with learned latency, frequency, width, and amplitude. $B\, a_t$ is a small set of fixed eye-dipole and neck-muscle topographies with free time courses. The nuisance term is there so that frontal cortical patches are not pushed into absorbing blinks.

Training minimizes

$$\mathcal{L} = \sum_{c \in M} \lVert x_c - \hat{x}_c \rVert_1 ,$$

where $M$ is a random set of electrodes hidden from the encoder and $x_c$ is the measured voltage on electrode $c$. We use the $\ell_1$ norm because 3D-PIUNet, which refines eLORETA source estimates with a 3D U-Net, found it more robust than squared error under noise.

The masked-electrode loss answers the objection raised by EEG2Rep. That paper argues that reconstructing raw signal wastes model capacity on artifacts, and responds by predicting in an abstract learned space. We keep a signal-level target but make noise that is independent across electrodes unpredictable by construction.

Research Program

The encoder is a lightweight spatiotemporal CNN in the style of ENIGMA. ENIGMA showed that a small 2D spatiotemporal CNN with per-subject linear alignment layers matches large transformers for EEG-to-image decoding. The encoder outputs, for each patch, a softmax weighting over cortical mesh vertices. This makes location and extent differentiable, because the patch's scalp pattern becomes a weighted sum of precomputed lead-field columns. It also outputs the parameters of each patch's time-course atoms and the nuisance time courses.

The lead field is not fixed to one template head. Data-driven forward model inference built 1,600 forward models from 16 MRIs and 100 skull conductivities, and compressed them into a 2D PCA space. It then selected a subject's model by a grid search on variational free energy, reducing synthetic localization error from 78.5 mm with a template head to 0.0 mm at 5 dB signal-to-noise ratio. We rebuild a comparable library from the 16 structural MRIs of the Henson & Wakeman dataset and interpolate smoothly in the PCA coordinates. We then treat $\theta$ as a per-subject latent shared across all of that subject's trials, fitted by gradient descent on $\mathcal{L}$. This turns their offline grid search into one differentiable variable inside an amortized network.

Following the hybrid scheme of Wong et al., who used a fixed sinc-pulse decoder for terahertz imaging and found that network initialization followed by classical local optimization reached deeper minima twice as fast as the optimizer alone, we allow a few test-time gradient steps per trial on $\mathcal{L}$ starting from the encoder's output. This addresses the stiff, non-convex landscape that fixed physical decoders are known to have.

The work draws on the other papers in the file in three ways.

  • Template for the method. The Bergman-model VAE of Wang et al. is the closest structural template. Its latents are the parameters of a physiological ODE that is integrated to reconstruct noisy glucose traces, and the resulting codes separated 29 of 33 patients on a 2D boundary. That model is purely temporal, however, and our contribution is the spatial forward map and the capacity argument.
  • Motivation. 3D-PIUNet needed 20-trial averages on THINGS-EEG2 and never re-projects its sources onto the scalp. That gap is what we target.
  • Constraint. Field Interpolation showed that a Maxwell-based forward model mapping arbitrary electrodes onto a fixed template enables leave-one-dataset-out transfer. Because our decoder works from electrode coordinates, it also handles arbitrary montages. The difference is that ours is learned end to end with source dynamics rather than applied as a static scalp-to-scalp map.

The quantity we analyze throughout is noise capacity: the fraction of variance the trained model can reconstruct on held-out segments known to contain no stimulus-locked brain signal. These are pre-stimulus baselines, recorded EMG bursts, and sensor noise synthesized from 1/f processes. The central prediction is that noise capacity falls steeply as $K$ shrinks and as the time-course family narrows, while the stimulus-evoked variance recovered on masked electrodes falls much more slowly. An unconstrained decoder of equal rank should show no such gap.

Baselines

The naive reference is the same downstream decoder trained on raw preprocessed EEG, run by the released code for ATM and ENIGMA with their default settings. ATM projects EEG into CLIP space and reported about 28.6% Top-1 zero-shot 200-way retrieval on THINGS-EEG. Our method enters only as a front end: the downstream decoder receives either our physically reconstructed scalp signal or our source codes. Every other part of the pipeline stays identical.

The strongest published retrieval method is ViEEG, which encodes contour, object, and scene streams hierarchically. It reports 40.9% Top-1 subject-dependent and 22.9% leave-one-subject-out 200-way retrieval on THINGS-EEG. It is the strongest competitor because its gains come from the image side, leaving the EEG side as the untouched bottleneck that we address. We will run ViEEG with and without our front end under its published protocol and hyperparameters. Beating it means a statistically reliable Top-1 gain over subjects in the leave-one-subject-out setting, where mismatched head shapes should make the per-subject $\theta$ matter most.

For consumer hardware, ENIGMA reports 86% human two-alternative forced-choice identification on THINGS-EEG2 and 71% on the 32-channel, 250 Hz Alljoined-1.6M. Our success criterion there is a larger gain on Alljoined than on THINGS-EEG2, because noise is heavier there. We will use an automated forced-choice proxy with a fixed image-embedding model, since running new human raters would not be comparable to their numbers.

Single-trial test inputs are the regime this proposal is really about. We will evaluate them alongside the published protocols, but no reported result found on Lacuna for single-trial THINGS-EEG2 retrieval.

The decisive ablation replaces $L_\theta$ with a learned, unconstrained linear decoder of the same rank $K$. It uses the same time-course family, the same masked-electrode loss, the same encoder, and the same training budget, and it amounts to a learned ICA-like autoencoder. A second ablation keeps the lead field but allows a distributed source code across all mesh vertices, which tests the capacity argument directly. A third removes the nuisance term. A fourth fixes $\theta$ to the template head.

For source localization, the baselines are eLORETA, 3D-PIUNet trained on its published synthetic setup, and the variational-garrote method of Data-driven forward model inference on Henson & Wakeman. For motor-imagery transfer, the baselines are the Riemannian tangent-space pipelines from the MOABB study, which "frequently exceeded 0.80 AUC" while EEGNet-style networks sat at 0.60 to 0.70, and Field Interpolation plus Riemannian re-centering in the leave-one-dataset-out setting. Exact accuracies for Field Interpolation: no reported result found on Lacuna.

Why not just…?

Why not just apply a surface Laplacian before the decoder? The Laplacian (current source density) is a fixed spatial high-pass filter that sharpens focal activity (Kim et al.). It suppresses spatially broad noise but amplifies electrode-local noise, and it degrades with sparse or irregular montages (Xu et al.), which is exactly the Alljoined case. It is cheap, so we include it as a front end to ATM and ViEEG under identical settings. If it matches our model on Alljoined, our denoising claim fails there.

Why not just learn a low-rank autoencoder, or run ICA, with no physics? This is the real threat, and it is the decisive ablation above. A rank-$K$ learned decoder can place its spatial patterns anywhere in the 64-dimensional sensor space, including onto muscle and electrode-drift topographies. The lead-field version can reach only patterns generated by compact cortical patches. If the learned decoder matches ours on noise capacity and downstream retrieval, we will state that the physics adds interpretability and cross-montage transfer but not denoising, and restrict the claim to those.

Why not just run eLORETA, or 3D-PIUNet, and decode the estimated sources? Distributed inverse solutions have no capacity limit, so they pass noise into source space. 3D-PIUNet improved over eLORETA on synthetic data but was trained on synthetic Gaussian sources and applied to 20-trial averages. Our noise-capacity measurement and single-trial retrieval comparison with eLORETA and 3D-PIUNet features will show whether a closed-loop, few-source model does better per trial.

Why not just use Riemannian covariance pipelines, or scale a pretrained model like LaBraM? For motor imagery, Riemannian methods are the bar to clear, and MOABB shows deep networks need more than 150 trials per class to compete. We will report an honest leave-one-dataset-out comparison, and a tie there would be acceptable. For visual retrieval, covariance matrices discard the 200 to 400 ms temporal structure that ATM identifies as decisive. LaBraM (TUAB AUROC 0.9162) is a complementary encoder rather than a rival. We will test whether its features, fed through our lead-field bottleneck, beat LaBraM features used directly.

Proposed Experiments

The first check is physical and runs on Henson & Wakeman. Starting from the template, does gradient descent on $\theta$ recover coordinates close to each subject's own MRI-based forward model? Does the few-source model localize the N170 near 160 ms to the fusiform region on single trials, where the variational-garrote method used averages? A matched simulation at 0 to 30 dB with real EMG and blink recordings added will compare localization error against the 78.5 mm template and 0.0 mm fitted-model figures reported by Data-driven forward model inference.

The second check is the noise-capacity curve. We will plot noise capacity and evoked-signal recovery against $K$ and the size of the time-course family, for the lead-field decoder and the unconstrained decoder. The prediction is that only the lead-field decoder separates the two curves. We will also measure the inverse relationship: whether patches with small softmax extent correspond to low noise capacity.

The third check is downstream. We will measure zero-shot retrieval on THINGS-EEG2 under published protocols and on single trials, in subject-dependent and leave-one-subject-out splits, with ATM, ENIGMA, and ViEEG decoders each run with and without our front end. We will add forced-choice evaluation on Alljoined-1.6M. We will also examine ViEEG's reported failure where faces are generated for cat images, by checking whether retrievals driven by patches outside visual cortex decrease once those patches are down-weighted.

The fourth check is leave-one-dataset-out motor imagery on the six MOABB datasets used by Field Interpolation, including Shin2017A with its few shared channels.

The proposal succeeds if the lead-field model beats the equal-rank learned decoder on single-trial retrieval and on Alljoined. It narrows to an interpretability and transfer contribution if it only matches that decoder.