Noise-Aware Spectral Initialization of Adapters for EEG Foundation Models

Brain foundation models are large networks pretrained on thousands of hours of scalp electroencephalography (EEG), the electrical activity recorded through electrodes on the head. To use one of these models for a new task or a new person, practitioners either fine-tune every weight or train a small head on top. A cheaper option from language modeling is the low-rank adapter (LoRA). LoRA freezes each weight matrix and learns a small correction to it, the product of two thin matrices. Recent work shows that where those thin matrices start matters a great deal. The best-performing starting points use a singular value decomposition, a way of splitting a matrix into ranked directions, applied to the weights, the activations, or the first gradients.

All of these recipes assume the adaptation data are clean. EEG is not. The largest-variance directions in EEG activations are often eye blinks, muscle bursts, and sensor drift rather than brain responses. We propose to start adapters in the directions of the network's activation space where task-related variation is largest relative to trial-to-trial noise, not where total variation is largest. The method is a generalized eigenvalue problem between two covariance matrices: one for between-condition activations and one for within-condition activations. We will test whether this starting point beats variance-based and gradient-based spectral initialization, and whether it narrows the gap between decoding people seen in training and decoding new people, as measured on THINGS-EEG2 and on consumer-grade Alljoined-1.6M.

Core Claim

Existing data-driven initialization is fooled by high-amplitude noise, and there is a cheap fix. Explained Variance Adaptation (EVA) runs incremental SVD on a layer's activation vectors over a calibration batch. It sets the adapter's input projection to the top right-singular vectors and uses explained-variance ratios to redistribute rank across layers. That choice provably maximizes the expected gradient norm at initialization. Under low signal-to-noise ratio (SNR), however, the gradient norm is maximized along artifact directions. LoRA-GA instead takes the SVD of a single-batch weight gradient, which with a few dozen noisy EEG trials is itself dominated by noise.

Our claim is that EEG labels supply a principled contrast that neither method uses. Activations should vary across stimuli or classes (the signal we want) and should be stable across repeated presentations of the same stimulus (so remaining variation is noise). Initializing the adapter in the subspace that maximizes this ratio should give three things relative to EVA and LoRA-GA: faster convergence, higher final accuracy, and a much smaller accuracy drop as artifact power increases. The same quantity, per-layer generalized eigenvalues, also gives an SNR-based rule for allocating rank across layers. That rule can be compared directly with EVA's variance-based rule on an architecture that separates spatial from temporal processing.

This is not a benchmark of PiSSA versus MiLoRA versus EVA on EEG, although we run that comparison as a baseline. The new ingredient is the noise-normalized initialization and the rank-allocation rule derived from it.

Method

For a frozen weight matrix $W \in \mathbb{R}^{d \times k}$ inside an EEG encoder, collect calibration activations $x \in \mathbb{R}^{k}$ at that layer's input. Group the trials by condition $c$, which is stimulus identity on THINGS-EEG2 (each image is shown repeatedly) or class label on classification benchmarks.

Let $\Sigma_s$ be the covariance of the condition means $\bar{x}_c$ across conditions. Let $\Sigma_n$ be the pooled within-condition covariance of $x - \bar{x}_c$. The adapter's input matrix $A \in \mathbb{R}^{r \times k}$ is initialized from the top $r$ generalized eigenvectors of

$$\Sigma_s v = \lambda\, (\Sigma_n + \epsilon I)\, v,$$

where $\lambda$ is the ratio of signal variance to noise variance along direction $v$ and $\epsilon$ is a shrinkage term. $\epsilon$ is needed because a single subject's calibration set may hold only a few trials per condition. As in LoRA, the output matrix $B$ starts at zero, so the pretrained model's outputs are unchanged at step zero. The scale of $A$ follows EVA's convention, which keeps the comparison controlled.

This is the same mathematics as classical EEG spatial filters that maximize evoked-to-background power. The difference is where it is applied: to intermediate activations of a pretrained Transformer, as an adapter initialization, rather than to raw electrodes as a fixed preprocessing step.

For rank allocation, each layer $\ell$ gets rank in proportion to how many of its generalized eigenvalues $\lambda^{(\ell)}_i$ exceed a threshold. This replaces EVA's explained-variance ratio with an SNR count. On CBraMod, which runs separate spatial-attention and temporal-attention branches in parallel, this directly asks which branch carries more decodable, repeatable structure for a given task.

We will also report, for each layer, how the chosen directions project onto the singular spectrum of $W$. The tension between PiSSA and MiLoRA motivates this analysis. PiSSA initializes the adapter in the top singular directions of $W$ and freezes the rest as residual. MiLoRA adapts only the minor directions to limit forgetting. Neither paper addresses noisy inputs. Measuring where high-SNR directions fall in the weight spectrum gives an empirical answer to which subspace matters for noisy data, rather than an assumed one.

Research Program

Backbones and code. We build on the public LaBraM code and CBraMod code.

  • LaBraM is a Transformer with 5.8M to 369M parameters. It tokenizes one-second single-channel patches by their Fourier amplitude and phase and is adapted by full fine-tuning or linear heads.
  • CBraMod was pretrained on about 9,000 hours of Temple University Hospital EEG and is adapted by full fine-tuning.

Neither paper tests any parameter-efficient adaptation, so our study supplies that missing comparison. It also tests whether noise-aware starts matter specifically in the regime these papers flag, where residual noise remains even after pretraining on cleaned data.

Tasks. - Visual decoding. On THINGS-EEG2 and Alljoined-1.6M, we attach a CLIP-projection head to each backbone. We train with the contrastive image–EEG objective used by ATM and ViEEG, and place adapters in every attention and feed-forward projection. - Clinical and BCI classification. On TUEV, TUAB, PhysioNet-MI, and FACED, conditions are class labels.

Evaluation regimes. - Task adaptation. One shared adapter is trained on pooled subjects. - Subject calibration. A per-subject adapter is initialized only from that subject's calibration data. Budgets are 5, 15, and 30 minutes, matching the budget ENIGMA made standard.

Analysis products. - For each method, we track how much of the adapter's initial input subspace overlaps with known artifact directions. These directions are estimated from EOG-correlated and high-frequency EMG-band activity in the same activations. - We track how that overlap evolves during training. - We report the spatial-versus-temporal rank split chosen by the SNR rule on CBraMod.

Baselines

Frozen backbone with a linear head. This is the naive reference and the evaluation LaBraM itself reports. It shows how much any backbone adaptation buys under this data regime. It uses the same heads, data splits, and early-stopping protocol as every other arm.

Full fine-tuning. This is the default in both LaBraM and CBraMod, and it is feasible at these model sizes. It is the strongest competitor on accuracy when the adaptation data are large.

  • Reported numbers. LaBraM-Huge reports balanced accuracy 0.6616 on TUEV and AUROC 0.9162 on TUAB (LaBraM). CBraMod reports balanced accuracy 64.17% on PhysioNet-MI and Cohen's kappa 0.5041 on FACED (CBraMod).
  • Fair comparison. We reproduce these numbers with the released code before comparing. Every arm gets the same learning-rate and epoch search budget.
  • What counts as beating it. We expect to match full fine-tuning on the large clinical sets and beat it in the 5–15 minute per-subject regime.

Vanilla LoRA. This uses random input matrix and zero output matrix. It isolates the effect of any spectral starting point.

The spectral family. This group comprises PiSSA, MiLoRA, LoRA-GA, and EVA with EVA's own rank redistribution. EVA is the strongest existing method on our exact question, because it is the only one that looks at the downstream data distribution; beating EVA is the central requirement.

  • Fair comparison. All four methods use the same total adapter parameter count, the same calibration trials as our method, and the same tuning grid. LoRA-GA is given its gradient averaged over the full calibration set rather than a single batch, so that it is not handicapped.
  • What counts as a win. We need statistically reliable gains over EVA across subjects on THINGS-EEG2 and Alljoined-1.6M, plus a smaller slope of accuracy against injected artifact power.

Visual-decoding reference numbers. These come from different architectures, so they are context rather than head-to-head comparisons. For 200-way zero-shot Top-1 retrieval on THINGS-EEG2, ViEEG reports 40.9% subject-dependent and 22.9% leave-one-subject-out (ViEEG). ATM reports approximately 28.6% (ATM). ENIGMA reports a human two-alternative-choice rate of 86% on THINGS-EEG2 and 71% on Alljoined-1.6M (ENIGMA). No reported result was found on Lacuna for any parameter-efficient adapter on an EEG foundation model.

Two ablations isolate the key ingredient. - Signal-only. EVA-style top eigenvectors of $\Sigma_s$ alone, with no noise normalization. This checks whether averaging over repeats is all that matters. - Noise-normalized total variance. Top eigenvectors of total covariance whitened by $\Sigma_n$. This checks whether the label contrast matters.

We also swap the two rank-allocation rules while holding the initialization fixed.

Why not just…?

Why not just clean the EEG first and then use EVA? This is the most serious threat. If standard artifact removal makes the top-variance activation directions task-relevant, our contribution shrinks to "preprocess carefully." We therefore run EVA on artifact-cleaned inputs as an extra arm. Our expectation that cleaning is insufficient rests on two points. First, CBraMod's pretraining data were already automatically cleaned, and the authors still report residual noise. Second, consumer recordings such as Alljoined-1.6M contain noise that cleaning cannot fully separate from signal. If cleaned-input EVA matches our method, we will report that plainly.

Why not just use ENIGMA-style per-subject linear alignment layers? They are cheap and calibrate in 15 minutes. However, they cannot change intermediate representations. ReAlnet needed alignment across multiple layers of a vision network to match EEG dynamics. ENIGMA's own human-rater score falls from 86% to 71% on noisier consumer hardware. We implement a per-subject input alignment layer on the same backbone as a baseline, and also combine it with our adapter, to see whether the two are complementary.

Why not just average LoRA-GA's gradient over more trials? Averaging does denoise the gradient, and we give LoRA-GA exactly that advantage. The remaining difference is that an averaged gradient still weights directions by total loss sensitivity, not by repeatability. The artifact-injection sweep will show whether that difference matters in practice.

Why not just fully fine-tune, since these models are small? Compute is not the concern; overfitting on tiny, noisy per-subject sets is. The drop from 40.9% to 22.9% when ViEEG decodes unseen subjects shows how badly idiosyncratic noise transfers. The calibration-budget curves will show whether full fine-tuning catches up as data grow. We expect it to at 30 minutes or more.

Decisive Checks

Controlled noise injection. On THINGS-EEG2 we add increasing power of realistic artifacts: recorded EOG and EMG templates projected through the montage, plus the Gaussian and DC-shift corruptions that EEG2Rep used to test robustness. The prediction is falsifiable. The artifact overlap of EVA's initial subspace should rise with artifact power, and ours should stay flat. The retrieval gap between the two methods should widen with noise. If neither happens, the premise is wrong.

Cross-subject calibration. A leave-one-subject-out protocol on THINGS-EEG2 and Alljoined-1.6M tests whether initializing per-subject adapters from 15 minutes of the new person's data recovers a meaningful share of the subject-dependent versus leave-one-subject-out gap. The comparison is against EVA, LoRA-GA, and linear alignment given identical data.

Principal versus minor directions. The weight-spectrum analysis reports whether the high-SNR directions sit in the principal subspace, the minor subspace, or neither. That result bears directly on whether PiSSA or MiLoRA is the right default for noisy biosignals.