In a brain-computer interface (BCI), meaning a system that turns recorded brain activity into commands, two learners adapt at once. The decoder updates to fit the user's brain signals. The user, whose neural patterns drift with practice and fatigue, learns to produce whatever the decoder rewards. This mutual adaptation is the reason BCIs work at all, and it is also why they break. If the decoder changes too fast, the user cannot form a stable picture of what it wants. If it never changes, it falls behind the user's drifting signals.
We propose to settle this tension by splitting the decoder in two. The front half is a learned vocabulary of discrete "brain tokens": short, recurring waveform patterns, each assigned an integer ID by vector quantization, which means each signal window is snapped to its nearest entry in a learned dictionary of prototype vectors. The back half reads only token IDs. Our claim is that the IDs, the meaning of "token 417," can be held fixed and act as a contract between human and machine. Meanwhile the prototype vectors behind each ID are allowed to move slowly with the user's physiological drift. Everything downstream then stays valid without recalibration: the command classifier, a frozen projector into a language model, and the user's learned mental strategy. Only the mapping from raw signal to ID adapts.
Core Claim
Existing electroencephalogram (EEG, scalp-recorded brain voltage) tokenizers learn their dictionary once, offline, and then freeze it. Tokenizing Single-Channel EEG with Time-Frequency Motif Learning builds an 8,192-entry dictionary of physiological motifs, and TempoGPT feeds dictionary tokens to a language model. The research file explicitly lists this static-dictionary assumption as an open limitation when users drift. Work on co-adaptive BCIs, on the other hand, adapts continuous decoder weights. Collinger et al. alternate decoder updates with fixed periods. First Contact tunes interface parameters by maximizing mutual information between user commands and state transitions. A Human-Machine Joint Learning Framework to Boost Endogenous BCI Training gives the user discrete "copy/new" prompts while the decoder learns with a curriculum. None of these separates what the decoder means from how it recognizes it.
We hypothesize that this separation, which a discrete vocabulary makes natural, delivers three things. First, it achieves cross-session accuracy comparable to supervised per-session recalibration without labeled recalibration data. Second, it keeps downstream modules, including a cross-modal projector into a language model, valid across weeks without retraining. Third, it gives users a more stable target, which should help most for the 15–30% of users who never reach functional control, as reported by Wang et al..
Mechanism
A pretrained encoder, initialized from the dual time-frequency encoder of the TFM-Tokenizer, maps each signal window to a vector $z$. The window is assigned the ID $k$ of the nearest prototype $e_k$. Downstream modules never see $e_k$. They see only $k$, through their own frozen embedding table. Between sessions $s$ and $s+1$, prototypes are updated without labels:
$$e_k^{s+1} = e_k^{s} + \min\!\left(\alpha,\ \frac{\rho}{\lVert \bar z_k^{s} - e_k^{s}\rVert}\right)(\bar z_k^{s} - e_k^{s}),$$
Here $\bar z_k^{s}$ is the mean encoder output assigned to ID $k$ in session $s$, $\alpha$ is a moving-average rate, and $\rho$ caps how far any prototype can move per session. The cap is a direct response to the instability warning from Wu et al. that fast decoder changes destroy the user's mental model.
The main risk is identity swapping: prototype $k$ could drift into the region that $j$ used to cover, silently changing what the user's strategy means. We guard against this in two ways. Updates go through a matching step that tracks clusters by temporal continuity. A handful of labeled "anchor" trials per session (we will report how few suffice) also check that each ID still predicts the same command distribution.
The co-evolution literature supplies a second constraint. How Algorithmic Confounding in Recommendation Systems Increases Homogeneity and Decreases Utility shows that retraining on behavior the system itself shaped narrows that behavior over time. Our tokenizer is retrained on signals the user produced in response to the tokenizer, so the same collapse could shrink the active vocabulary. We will track the entropy of token usage across sessions as a homogenization diagnostic. Following the measurement framing of Accounting for AI and Users Shaping One Another, we will also treat the token-usage histogram as an observable trace of the user's hidden adaptation state. This responds to that literature's noted lack of continuous physiological signals inside the human-AI loop.
Research Program
We will build the adaptive tokenizer, a lightweight ID-based command head, and a frozen projector that maps token IDs into a small language model. The projector uses prefix-style conditioning as in Multimodal Few-Shot Learning with Frozen Language Models, and is trained once, early, in isolation, as Scalable Vision Language Model Training via High Quality Data Curation recommends. The language model plays a deliberately secondary role. It is a demanding downstream consumer that tests whether the contract holds. We do not claim that motor-imagery BCIs need a language model. Video-LLaVA shows that misaligned inputs to a shared projector hurt the language model. Our setting is a temporal version of that problem: one projector receiving inputs whose geometry drifts. Feeding IDs rather than prototype vectors is our answer to it.
Analysis runs in three stages. First, offline multi-session replay on the IDIAP EEG data from EEG-based BCI Systems and IDIAP EEG Database, which records asynchronous motor imagery across sessions and therefore exposes real drift. Second, simulated co-adaptive users. A model user moves its signal distribution toward whatever earned reward under the current decoder, so we can impose known amounts of drift and strategy change. Third, a small online study with naive users, stratified by early performance so that we can study poor performers separately.
Baselines
The naive reference is a decoder trained on session one and never updated. It sets the floor for how much drift costs.
The strongest practical competitor is supervised per-session recalibration of a standard decoder. This means common spatial patterns with a support vector machine, plus the compact networks EEGNet and the multiscale filter-bank network of A Parallel Multiscale Filter Bank Convolutional Neural Networks for Motor Imagery EEG Classification. Each gets a full labeled calibration block every session. It is strong because it has more labels than our method. We will run it on identical sessions with the same hyperparameter-search budget. We count our method as beating it if it matches its cross-session accuracy within confidence intervals while using at most a tenth of its labeled trials. Failing to match it would weaken the claim but would not refute it. Search on Lacuna found no reported cross-session result on the IDIAP data for any of these decoders, so all numbers will be produced by us under a shared protocol.
For co-adaptation in the online study, we compare against alternating updates in the style of Collinger et al. and against the joint-learning framework of Wang et al.. Search on Lacuna found no reported accuracy or information-transfer-rate number for the latter, beyond the 15–30% illiteracy figure it cites. Success means a higher information transfer rate in bits per second, the measure used by Chiappa and Millán, and a smaller fraction of users who fail to reach control.
Three ablations isolate the key ingredient. In the first, the dictionary is frozen and only the command head is retrained, which is the TFM-Tokenizer used as designed. In the second, the dictionary is retrained freely with no identity matching and no displacement cap. In the third, the continuous encoder output is adapted by the same moving-average rule with no discretization, and downstream modules read continuous vectors. The third ablation answers directly whether discreteness is what buys stability.
Why not just…?
Why not just recalibrate a standard decoder every session? This is the real threat, and we do not argue it away. Per-session recalibration works, but it costs a labeled block every session. It also invalidates every downstream module, and it changes the decision boundary the user has learned to target, which Blankertz et al. show users must track. The offline comparison settles accuracy per labeled trial. The online study settles whether boundary stability matters to the human. If recalibration wins on both, the discrete contract is not worth its complexity.
Why not just use unsupervised mutual-information adaptation as in First Contact? Reddy tunes a low-dimensional interface mapping with Bayesian optimization, which removes labels just as we do. Nothing in that method, however, keeps a learned mapping's meaning fixed while its inputs drift. Its search space also grows with the number of parameters. We will include a mutual-information-tuned mapping on top of the frozen tokenizer as a further comparison. If it matches our method, the right conclusion is that the adaptation rule matters less than the tokenized front end.
Why not just slowly adapt continuous features without any dictionary? This is the third ablation. Continuous slow adaptation also limits how fast the decoder changes. Its weakness is that it gives downstream consumers nothing invariant to hold onto. We predict that the frozen projector and command head degrade under continuous drift but not under ID-based input. If they do not degrade, the discreteness claim fails.
Proposed Experiments and Decisive Checks
The decisive offline check is downstream invariance. We train the command head and the language-model projector on session one, never touch them again, and measure accuracy, plus agreement of the projector's outputs with session-one outputs, across all later sessions. This is done under each adaptation scheme. Identity stability is measured as the fraction of IDs whose anchor-trial command distribution stays within a fixed divergence of its session-one value. Usage entropy is tracked to detect the homogenization predicted by the confounding literature.
In simulation, we sweep drift magnitude and the displacement cap $\rho$ to find where adaptation becomes destabilizing. This gives an empirical stability boundary that the online study can check.
The decisive online check compares information transfer rate and the fraction of non-performing users between our method and the two co-adaptation baselines over repeated sessions. Users are blinded to condition. Analysis is stratified by session-one performance, because the strongest prediction is a benefit concentrated in initially poor performers.
A clean negative result would also be informative. One example is identity swaps that no matching step can prevent. Another is vocabulary collapse even with the cap in place. Either would show that offline-trained discrete vocabularies cannot serve as a shared language in a closed loop.
References
- Tokenizing Single-Channel EEG with Time-Frequency Motif Learning
- TempoGPT: Enhancing time series reasoning in multi-modal language model via quantizing embedding
- An Electrocorticographic Brain Interface in an Individual with Tetraplegia
- First Contact: Unsupervised Human-Machine Co-Adaptation via Mutual Information Maximization
- A Human-Machine Joint Learning Framework to Boost Endogenous BCI Training
- A Parallel Multiscale Filter Bank Convolutional Neural Networks for Motor Imagery EEG Classification
- How algorithmic confounding in recommendation systems increases homogeneity and decreases utility
- Accounting for AI and Users Shaping One Another: The Role of Mathematical Models
- Multimodal Few-Shot Learning with Frozen Language Models
- Scalable Vision Language Model Training via High Quality Data Curation
- Video-LLaVA: Learning United Visual Representation by Alignment Before Projection
- EEG-based BCI Systems and IDIAP EEG Database
- The Berlin Brain–Computer Interface: Non-Medical Uses of BCI Technology