Hospitals are beginning to hand raw brain-voltage recordings directly to large language models. Systems like CELM feed hundreds of EEG channels — the scalp voltages recorded from electrodes on the head — through a neural encoder and into an instruction-following LLM that writes the clinical report, including diagnostic impressions and abnormality descriptions (Neural Signals Generate Clinical Notes in the Wild). The security literature has established that language models cannot reliably tell the difference between their operator's instructions and instructions smuggled inside untrusted data — the prompt injection problem — and that this failure applies to text in retrieved documents, images, web pages, and audio (Formalizing and Benchmarking Prompt Injection Attacks and Defenses, Manipulating Multimodal Agents via Cross-Modal Prompt Injection). Nobody has asked the obvious and increasingly urgent question: can a malicious instruction be carried by the brain-signal channel itself?
This proposal treats the continuous EEG recording as an untrusted data channel into an LLM pipeline, and builds the first attack, benchmark, and defense study for what we call neural-channel prompt injection. The core claim is two-part. First, an adversary who controls the signal path between the patient's scalp and the model — compromised acquisition firmware, a tampered amplifier, or a hostile device driver — can optimize a small perturbation over the EEG time series so that, after the encoder's tokenization and projection into the LLM's input space, the resulting token embeddings act as a prompt injection that overrides the clinical system instructions. Second, whether the attack survives depends critically on an untested design choice: continuous epoch-token embedding (as in CELM) versus discrete quantization through a vector-quantized codebook (as in DeWave), meaning quantization may be a cheap structural defense or a trivially bypassed one — and we do not currently know which.
Core Claim and Why It Is Plausible
The plausibility argument comes from two directions in the file. On the attack side, CrossInject showed that continuous sensor perturbations optimized to align with target semantics in a surrogate encoder's latent space become structured commands once mapped into the language model, raising attack success rate by an average of 30.1% over text-only baselines, while purely unimodal attacks scored 0% (Manipulating Multimodal Agents via Cross-Modal Prompt Injection). EEG-to-language pipelines present exactly the alignment problem CrossInject exploits — an encoder trained to project sensor signals into token-space semantics — but with a crucial aggravation: in zero-context or resting-state operation there is no second modality to cross-check the neural representation, the very mechanism that defeated unimodal attacks in the vision setting. On the capacity side, DeWave demonstrates that EEG projected into a discrete codebook aligned with word embeddings supports open-vocabulary generation at 41.35 BLEU-1 with eye-tracking markers and 20.5 BLEU-1 on raw waves without them (DeWave: Discrete Encoding of EEG Waves for EEG to Text Translation). If clean EEG can be projected into fluent text, the encoder's latent space is semantically rich enough to be steered toward a chosen token sequence by an adversary.
The natural objection — EEG is noisy, volume-conducted, and artifact-ridden (Application of deconvolutional networks for feature interpretability in epilepsy detection) — cuts the other way: the same tokenization machinery that strips physiological noise also strips any barrier to adversarially chosen structure, because it is trained to find signal-like patterns, and an optimized perturbation is the most signal-like pattern an encoder can be shown. The empirical check of this claim is Experiment 1.
Threat Model
We assume an attacker who can modify the EEG data stream in transit or at the source: compromised electrode-amplifier firmware, a tampered bedside acquisition box, or malicious signal-processing middleware between the hospital recorder and the report-generation service. This is the same trust boundary that all indirect prompt injection work assumes for retrieved documents, extended to a sensor feed that no clinician reads token by token. The attacker does not know the patient's true brain state but can observe or replay typical recordings. The injection goal is not gibberish but a targeted clinical manipulation: suppressing the description of epileptiform abnormalities, upgrading a benign record to an urgent diagnosis, or exfiltrating patient identifiers from the electronic health record context embedded in the prompt. Resting-state and long-duration continuous recordings are the primary attack regime: unlike stimulus-locked paradigms such as ZuCo reading EEG (DeWave) or THINGS-EEG image viewing (RealMind: Advancing Visual Decoding and Language Interaction via EEG Signals), there is no external stimulus to verify the neural representation against, and the stakes are concrete — for example, the multiscale-entropy models used to distinguish persistent vegetative state from minimally conscious state in resting-state EEG (Identifying Features of Electroencephalography Associated with Improved Awareness in Persistent Vegetative State via Multiscale Entropy) directly inform consciousness-level decisions where a manipulated downstream report could change care.
Research Program
Part 1 — The attack (EEG-Inject). We adapt the latent-alignment strategy of CrossInject (Manipulating Multimodal Agents via Cross-Modal Prompt Injection) from 2D images to spatiotemporal voltage matrices. Given a clean recording $X \in \mathbb{R}^{C \times T}$, an EEG encoder and projector $E$, the system instruction $s$, and clinical context $c$, we solve
$$\delta^* = \arg\max_{\|\delta\| \le \epsilon} \; \log p_\theta\big(y_{\text{cmd}} \mid [\,s,\; E(X+\delta),\; c\,]\big)$$
where $\delta$ is the perturbation added to the signal, $\epsilon$ caps its size relative to typical artifact amplitudes (so the corrupted recording stays plausible to a visual inspection of the waveform), $\theta$ is the frozen report-generation LLM, and $y_{\text{cmd}}$ is the attacker's desired output, such as "no epileptiform abnormalities noted." Because $E$ is differentiable end-to-end in architectures like CELM, gradients flow from the LLM's output distribution back through the projector and encoder into the channel-time matrix. For quantized pipelines we optimize in two regimes: pre-quantization (straight-through gradients through the codebook lookup, following standard vector-quantization practice) and a discrete search over codebook entries, where the attacker swaps quantized tokens directly. We additionally test a coordinated attack that perturbs the EEG channel while injecting a benign-looking stylistic cue into the EHR text field, mirroring the bidirectional pressure that made CrossInject effective, and an EEG-only attack, mirroring the unimodal baseline that scored 0% in the vision setting — the comparison that determines whether the absence of a cross-checking modality is what makes the neural channel uniquely dangerous.
Part 2 — The benchmark (NeuroInject). We package the attack into a benchmark on two pipelines and datasets from the file: a CELM-reproduction on the Harvard EEG-Report benchmark (roughly 10,000 clinical reports paired with over 11,000 hours of continuous EEG across 10,000 patients, with reports generated for multi-hour recordings) and a DeWave-reproduction on ZuCo 1.0/2.0 (128-channel reading EEG with eye-tracking). Each item consists of a clean recording, a clinical context, a system instruction, and one of five injected tasks (suppress abnormality, fabricate abnormality, exfiltrate EHR identifiers, override report format to embed a hidden instruction for downstream agents, and alter an impression of consciousness level on resting-state segments). We report attack success rate judged by a rubric-based grader, detection AUROC for each defense, and utility degradation under clean inputs, adopting the ASV / PNA-T methodology of the formalized injection benchmark (Formalizing and Benchmarking Prompt Injection Attacks and Defenses) so results are directly comparable to the text-injection literature, where the average attack success value across 10 LLMs and 7 tasks was 0.62.
Part 3 — The defenses. We test three existing defenses for transfer to the neural channel, and build one new one. First, Attention Tracker's Focus Score — a training-free detector that flags injections by measuring attention diversion away from system-instruction tokens across fewer than 1% of "important heads," achieving AUROC of 1.00 on several models in the text setting (Attention Tracker: Detecting Prompt Injection Attacks in LLMs) — applied unchanged to EEG-token streams: if the same heads track the system prompt when hundreds of epoch tokens intervene, the detector transfers; if calibration must be redone per encoder, the "distraction effect" is a text-specific phenomenon and the strongest cheap defense fails here. Second, DeTAM-style attention modification, which cut average attack success rate on LLaMA-2-7b-chat from 31.8% to 0.8% by amplifying attention on parsed core-intent tokens (DeTAM: Defending LLMs Against Jailbreak Attacks via Targeted Attention Modification), extended to amplify attention on the clinical system tokens when EEG tokens attract it. Third, Position-enhanced Fine-Tuning (PFT), which restores robustness by inserting a large positional index gap between system and user tokens and raised TensorTrust extraction resistance from 33% to 62% on Llama-3-8B (The Illusion of Role Separation: Hidden Shortcuts in LLM Role Learning (and How to Fix Them)) — tested at multi-hour token lengths, because its own authors show that perceived system-privilege decays with token index, and CELM-scale inputs generate token sequences far beyond anything PFT was validated on. The new defense is quantization gating: routing the neural channel through a discrete, low-cardinality codebook with a reconstruction-quality bound, so that any input whose codebook reconstruction error exceeds a threshold — as an injected perturbation designed in continuous space may — is rejected or re-encoded. Whether this works is precisely the Part-1 question of whether quantization is a barrier or a conduit.
Proposed Experiments
Experiment 1 (feasibility and severity) runs EEG-Inject against the CELM-style pipeline on the Harvard EEG-Report benchmark, measuring rubric attack success rate as a function of perturbation budget $\epsilon$ normalized to artifact amplitude, and reporting at which $\epsilon$ a board-certified-style visual inspection can distinguish corrupted from clean waveforms. Success at injection-undetectable $\epsilon$ on even one of the five tasks establishes the vulnerability. Experiment 2 (quantization as defense) repeats the attack on the DeWave-style VQ-VAE pipeline on ZuCo, comparing pre-quantization continuous injection, straight-through gradient injection, and direct codebook-entry substitution, against the continuous epoch-token pipeline; the decisive readout is the ratio of attack success through discrete versus continuous tokenization. Experiment 3 (defense transfer) runs the three existing defenses plus quantization gating on the benchmark, reporting detection AUROC, attack success rate, and clean-input utility (ROUGE-1 and METEOR against the ground-truth reports, matching the metrics used in the CELM evaluation). Experiment 4 (long-context erosion) tests PFT at EEG durations from minutes to the 3-hour maximum CELM supports, measuring injection success and system-instruction adherence as a function of token distance — the open question flagged by the role-separation authors themselves.
Baselines
The do-nothing reference point is the clean, undefended pipeline: the CELM-style model and the DeWave-style model run on unmodified inputs with no defense, which pins the utility ceiling (and the floor of zero protection — any injection that succeeds trivially there). For the attack side, the strongest competitor is not our EEG attack but the text-channel indirect injection: an adversarial instruction planted in the EHR context field, evaluated under the ASV methodology in which injections against undefended models averaged 0.62 attack success value across tasks and models (Formalizing and Benchmarking Prompt Injection Attacks and Defenses). Every experiment runs the text-channel attack and the EEG-channel attack on identical recordings, contexts, compute, and tuning budget, so the benchmark answers the comparative question a reviewer actually cares about: is the neural channel a weaker, equal, or stronger attack surface than the text it is paired with? For the defense side, the strongest existing method to beat is Jatmo, which hard-codes the task into a base model's weights via synthetic fine-tuning and achieved prompt injection success below 0.5% versus 87% for GPT-3.5-Turbo, while retaining approximately 98% of the teacher model's output quality (Jatmo: Prompt Injection Defense by Task-Specific Finetuning). We will train a Jatmo variant for single-section EEG report generation on the same data and tuning budget as our defended models, and we count a defense as competitive only if it matches Jatmo's injection resistance within a few points while surviving the test Jatmo's own authors concede it fails: multi-tier, multi-turn clinical reporting with free-text clinician follow-up. For detection, the strongest competitor is Attention Tracker with its reported AUROC of 1.00 on several models in the text setting (Attention Tracker); transferring it unchanged and retraining only its calibration threshold is the fair comparison, run on identical EEG-injection test sets. Finally, the key ablation isolating our proposal's central ingredient is continuous-versus-discrete tokenization: the same attack, the same LLM, the same recordings, routed through epoch-aggregated continuous embeddings versus a VQ-VAE codebook, so that any difference in attack success is attributable to the quantization step alone rather than to model scale or training data.
Why not just…?
Why not just filter and paraphrase the text inputs before they reach the model? This is the first thing a clinical deployment would try, and the file contains direct evidence against it. In the formalized benchmark, ten defenses were evaluated across ten LLMs and none eliminated injection, while input mutations like paraphrasing dropped clean-task performance (PNA-T) by an average of 0.14 — a cost that in clinical reporting could corrupt the precise terminology ("status epilepticus," "triphasic waves") the report exists to preserve (Formalizing and Benchmarking Prompt Injection Attacks and Defenses). More importantly, text filtering does nothing about the channel we study: a signal-borne injection never passes through a text field, so the defense is structurally blind to it. Our Experiment 1 versus the text-channel baseline makes this quantitative.
Why not just drop the LLM and use a classifier plus rule-based report templates? This fully removes the attack surface, and for single-label triage (seizure versus non-seizure, in the spirit of the CNN seizure-detection line) it is genuinely adequate. But the CELM authors document exactly why hospitals want more: template pipelines cannot produce the multi-level synthesis neurologists actually need, and CELM's foundation-model approach delivered roughly 70–95% relative improvement in ROUGE-1 and METEOR over such baselines, with human experts rating the generated reports as more clinically actionable (Neural Signals Generate Clinical Notes in the Wild). The honest position is that this alternative trades the vulnerability for a known, large utility loss, and our benchmark quantifies both sides by running a classify-then-template arm alongside the LLM arms on the same recordings. If the template arm's clinical utility were adequate for a given deployment, our results would say so — that is a finding, not a failure of the proposal.
Why not just use Jatmo-style task-specific fine-tuning and be done? Jatmo is the strongest single defense in the file — under 0.5% injection success at about 98% of teacher quality (Jatmo: Prompt Injection Defense by Task-Specific Finetuning) — and we treat it as the baseline to beat rather than a strawman. Two of its own limits apply directly here: its authors state it is built for single, fixed tasks and cannot support interactive or multi-section generation, whereas clinical EEG reporting is multi-tier and conversational; and it requires a non-instruction-tuned base model, which increasingly do not exist for modern multimodal clinical stacks. Our defense comparison runs Jatmo head-to-head under matched tuning budgets, so if it turns out Jatmo's single-task variant does survive multi-tier reporting with utility intact, the experiments will show that and the field should adopt it — we state this openly because it is a real possibility.
Decisive Checks and Interpretation
Three outcomes would each be decisive. If EEG-Inject succeeds at visually undetectable perturbation budgets on the continuous pipeline, neural-channel prompt injection is a real vulnerability class and every deployed EEG-to-LLM system inherits it. If attack success through the discrete VQ-VAE channel is dramatically lower than through continuous embeddings, quantization gating becomes a deployable structural defense and the field gains a design rule. If Attention Tracker's Focus Score transfers to EEG-token streams with AUROC near its text-setting 1.00 (Attention Tracker), there is a zero-training detector for the new channel; if it collapses, that itself is the first evidence that injection signatures are modality-dependent and text-only defenses cannot be assumed to cover biosignal inputs. No reported result exists on Lacuna for any attack or defense evaluated on EEG-borne injection, which is the gap this proposal fills.
References
- Neural Signals Generate Clinical Notes in the Wild
- Formalizing and Benchmarking Prompt Injection Attacks and Defenses
- Manipulating Multimodal Agents via Cross-Modal Prompt Injection
- DeWave: Discrete Encoding of EEG Waves for EEG to Text Translation
- Application of deconvolutional networks for feature interpretability in epilepsy detection
- RealMind: Advancing Visual Decoding and Language Interaction via EEG Signals
- Identifying Features of Electroencephalography Associated with Improved Awareness in Persistent Vegetative State via Multiscale Entropy: A Machine Learning Modeling Study
- Attention Tracker: Detecting Prompt Injection Attacks in LLMs
- DeTAM: Defending LLMs Against Jailbreak Attacks via Targeted Attention Modification
- The Illusion of Role Separation: Hidden Shortcuts in LLM Role Learning (and How to Fix Them)
- Jatmo: Prompt Injection Defense by Task-Specific Finetuning