Recent systems turn functional MRI (fMRI) scans, which measure blood-oxygen signals across tens of thousands of small brain volumes called voxels, into captions and answers to questions about what a person saw. They do this by feeding brain-derived tokens into a large language model. The reported scores are strong, but the language model attached to the brain signal knows a great deal about how scenes usually look. Dressers are usually brown, and kitchens usually contain ovens. A decoder can score well by reading a coarse hint from the brain, such as "indoor scene, furniture," and letting the language model fill in the rest. Vision-language research has shown that this happens with images, where text-only models solve many supposedly visual benchmarks. Brain decoding has not been audited in the same way. This matters because the field is now arguing about whether finer spatial resolution helps decoding, and that argument cannot be settled while brain-derived content and language-prior content are mixed into one score.
We propose to measure how much of each decoded answer can be traced to the brain signal, and to use that measurement to answer one concrete question: does dividing the brain into finer spatial units, moving from about 17 large networks down to single voxels, increase the information decoders draw from the brain, or does it only raise scores on questions a blind language model could already answer? The method borrows from the study of synaptic weights, the connection strengths between neurons. These cannot be measured directly in living animals. Researchers infer them from how observed activity varies together and check the inference against simulations in which the true weights are known. We treat a decoder's reliance on the brain the same way: as a hidden quantity estimated from how outputs change when brain inputs are perturbed, and validated on semi-synthetic brain data where we control which information is present.
Core Claim
Current fMRI-to-text scores cannot separate brain-derived from prior-derived content. Two measurements together make them separable. The first is a split of test items into "hard" items, where a text-only model is confidently wrong. The second is a per-token reliance estimate, computed from paired true-versus-swapped brain inputs and calibrated against planted ground truth. Using these measurements, we predict that finer spatial units improve hard-item accuracy only up to the point where voxel noise dominates. We also predict that much of the headline gain reported for large-language-model decoders comes from easy items. Both predictions are falsifiable. If hard-item accuracy rises steadily down to single voxels, the "signal smearing" view of coarse parcels is supported. If hard-item accuracy stays flat while overall scores rise, finer resolution has mostly given the language model a better cue to reach for its priors.
Where This Sits
Three lines of work motivate the audit.
First, vision-language research shows that blind models exploit benchmarks. Thomason et al. found that single-modality agents match or beat multimodal ones on navigation and embodied question answering. Wu et al. defined a "linguistic gap": overall accuracy minus accuracy on items where GPT-2 prefers the wrong caption. They showed that hard-negative fine-tuning made this gap grow from 3.43 to 13.24 even as accuracy rose. Chimera found blank-image accuracy well above chance, along with models that reason "better" than they recognize. LIME filters out text-solvable items. We reuse the Wu et al. hard-split construction directly. What we change is the domain, brain decoding, where no such audit appears in the file, and we add a calibrated per-token reliance measure that none of these works have.
Second, large-language-model brain decoders have not been audited this way.
- MindLLM encodes each voxel by its 3D coordinates and region label and then instruction-tunes Vicuna on about a million brain-grounded dialogues.
- BIT-LLM keeps brain tokens present through cross-attention to resist language priors. It reports that shuffling or zeroing voxels collapses performance.
- UMBRAE aligns brain signals to image-encoder features and reports captioning and object grounding.
- NEURONA answers compositional questions over Yeo-17 network parcels. Its authors state that scaling to individual voxels is the necessary next step.
- The cross-subject video decoder trains on captions generated by another model and fixes its "super-voxel" size without ablating it.
None of these papers tests a text-only baseline on its own question-answer benchmark. BIT-LLM's collapse result is the closest thing to an audit. It shows that the brain input is used for something, but it cannot show which parts of an answer depend on it.
Third, earlier encoding work constrains how we design the study. Robust Evaluation of Language-Brain Encoding showed that choosing voxels using test data inflates explained variance. It also showed that pairwise accuracy falls to chance on continuous narrative stimuli. We therefore fix every parcellation and voxel selection on training subjects and training data only.
The granularity literature supplies the competing hypothesis. Sub-region work on areas MT and MST (Huk et al.), on thalamic nuclei (Gohel et al.), and on voxel-wise association studies (Cheng et al.) argues that averaging over large regions cancels signals. A Spark-based pipeline study of Human Connectome Project data (Spark fMRI pipeline) found 1.5–2% higher task-classification accuracy with the 116-region AAL atlas than with the coarser Harvard-Oxford atlas. On the other side, single voxels have low signal-to-noise ratio. This disagreement is exactly what an uncontaminated metric could settle.
The synaptic-weight literature supplies the measurement logic. Lemke et al. inferred hidden, changing synaptic weights from sliding-window cross-covariance of spikes and validated the estimates against simulated ground truth. Simple cross-covariance outperformed noisier information-theoretic measures. Linderman et al. and Mehta et al. framed such hidden quantities as latent variables to be inferred. Mehta et al. also warned that different hidden mechanisms can produce nearly identical outputs. We take three lessons from this work. The reliance estimator should be a simple covariance-style statistic rather than an elaborate one. It must be validated on simulations with known ground truth. And identifiability, meaning whether brain reliance can be distinguished from prior use at all, must be checked rather than assumed.
Research Program
The first component is a blind-solvability audit of existing benchmarks. We fine-tune the same language model each decoder uses, Vicuna-7B for MindLLM-style systems, on the question and answer text alone for BOLD5000-QA, CNeuroMod-QA, and the Brain Instruction Tuning (BIT) tasks. For captioning on NSD (the Natural Scenes Dataset), we fine-tune it to output COCO-style captions conditioned only on the image's dataset split and the question prompt. A stronger variant receives one coarse label, the predicted COCO supercategory, decoded from the brain by a linear classifier. This "category-plus-prior" model is the explicit form of the shortcut we suspect.
The second component is the hard split. Following Wu et al., an item counts as hard when the blind model assigns higher likelihood to a wrong answer than to the correct one. For captions, we build distractors by swapping objects, attributes, or relations and mark the item hard if the language model prefers the distractor. The "brain gap" is overall accuracy minus hard accuracy, computed for every decoder.
The third component is a per-token reliance estimate. For each output token, we compare its log-probability under the true brain input with its average log-probability under brain inputs from other stimuli, holding the prompt and previously generated tokens fixed. Write $\ell_t(b)$ for the log-probability of token $t$ given brain input $b$. The reliance of token $t$ is
$$r_t = \ell_t(b_{\text{true}}) - \mathbb{E}_{b' \sim \text{other stimuli}}\, \ell_t(b').$$
This is the input-output analogue of the cross-covariance used to track synaptic weights. It is close to zero when a token comes from the language prior and positive when it depends on the specific scan. Plotting $r_t$ against token position also tests the "conditioning decay" concern, the worry that later tokens in long captions drift toward priors.
The fourth component is a semi-synthetic ground truth, our analogue of the simulated spiking networks. We fit a voxel-wise linear encoding model from image features to NSD responses, of the kind used as the Algonauts 2023 baseline. We then generate synthetic scans for edited images in which the correct answer contradicts the prior, such as an oven with an unusual color or an object placed out of its usual context. Real noise residuals are added at matched signal-to-noise ratio. In these scans the relevant information is present by construction, and we know which voxels carry it from the encoding weights. A decoder that answers by prior will fail, and $r_t$ should be high exactly on the planted tokens. This lets us check whether $r_t$ recovers known reliance, which mirrors how Lemke et al. validated cross-covariance against true weights.
The fifth component is a granularity sweep. MindLLM's coordinate-keyed design accepts any spatial grouping, so we train one fixed architecture at several resolutions: Yeo-17 networks, a roughly 100-region atlas, data-driven parcellations of about 400 and about 1,000 regions built on training subjects only, and single voxels. Training data, steps, and hyperparameter search budget are the same at every level. Each level is scored on overall accuracy, hard accuracy, brain gap, mean $r_t$, and accuracy on planted counterfactual scans.
Baselines
The naive reference point is the blind text-only model trained on each benchmark's question-answer pairs. It is the most important baseline, because a decoder must beat it on hard items to show it uses the brain at all. Precedent from Do 3D LLMs Really Understand 3D Spatial Relationships? suggests this baseline can match full multimodal systems. We found no reported result on Lacuna for a text-only baseline on BOLD5000-QA, CNeuroMod-QA, or BIT. The category-plus-prior model sits one step above the blind model and represents the specific shortcut hypothesis. If a decoder's hard accuracy does not exceed this model's, its brain use amounts to category detection.
The strongest existing methods are NEURONA for compositional question answering and BIT-LLM and UMBRAE for captioning. For NEURONA, the file reports only "up to a 47% relative improvement" over UMBRAE and SDRecon on BOLD5000-QA and CNeuroMod-QA, with no absolute accuracy. For captioning, BIT-LLM reports subject-held-out CIDEr of 34.90 against MindLLM's 13.22 (BIT-LLM), and UMBRAE reports CIDEr 61.06 on BrainHub (UMBRAE). BIT-LLM is the most important competitor because its design aims specifically at resisting language priors. If its brain gap is small, much of our concern about shortcuts is answered for the strongest current design. We will run released code or faithful reimplementations on the same subject splits, with the same language model where architectures allow it. Every system will be rescored with our hard split, brain gap, and $r_t$.
Two ablations isolate what is new. The first rescoring every system with the hard split alone, without $r_t$ or planted data. This shows whether the calibrated reliance measure adds anything beyond static filtering. The second runs the granularity sweep with standard metrics only. If standard metrics and hard metrics rank the granularity levels identically, the audit has not changed any conclusion, and we will report that result as such.
"Beating" the baselines means something different here than in a typical method paper. Our deliverable is a measurement. It succeeds if $r_t$ correlates with planted reliance on semi-synthetic scans clearly better than shuffle-collapse or attention-weight attribution does. It also succeeds if hard-split rankings of decoders or granularity levels differ from overall-score rankings in at least one case that matters, such as the 17-network versus voxel comparison NEURONA raised.
Why not just…?
Why not just shuffle or zero the brain input, as BIT-LLM already does? A collapse under shuffling shows that a decoder uses the brain for something. It cannot show that the brain determines which object, attribute, or relation appears in the output. A decoder that reads "kitchen" from the brain and then writes a typical kitchen caption would also collapse when given a scan of a beach. Our category-plus-prior baseline is built to imitate exactly this behavior. If it nearly matches BIT-LLM on hard items, shuffle tests were not sufficient. If BIT-LLM clearly beats it, BIT-LLM's design is vindicated, and that result is still informative.
Why not just filter out text-solvable items, LIME-style, and report scores on the rest? This is a real threat to the novelty of the measurement, and the hard split is essentially this filter. We adopt it rather than argue against it. Filtering has two weaknesses. It discards most of the data, and it scores whole items when captions mix brain-derived and prior-derived tokens. The first ablation in Baselines tests directly whether the per-token measure and the planted ground truth change any conclusion. If they do not, the contribution reduces to applying filtering to brain benchmarks plus the granularity result, and we will say so.
Why not just run decoders at voxel level and report standard metrics, as NEURONA suggests? That is the experiment this proposal is built to correct. The Spark pipeline result shows that granularity changes accuracy. However, a gain in caption metrics can come from a sharper category cue that triggers priors more reliably. Only the hard split and planted counterfactuals tell whether finer spatial units deliver information the language model could not have guessed.
Why not just use attention maps or gradient attributions over brain tokens? Attention is not a calibrated measure of reliance. The synaptic-weight literature offers a parallel warning: Lemke et al. found that more elaborate estimators added noise compared with simple covariance. We include attention and gradient attribution as competing estimators in the planted-ground-truth test, which settles which one tracks known reliance.
Decisive Checks
The first check validates the estimator. On semi-synthetic NSD scans, $r_t$ must rank planted tokens above prior-consistent tokens and must beat attention and gradient attribution at recovering planted reliance. If it fails, the remaining analyses would rest on an unvalidated measure, and we would fall back to item-level hard splits only.
The second check is identifiability, following the sloppiness concern raised by Mehta et al.. We verify that two deliberately constructed decoders produce distinguishable $r_t$ profiles even when their overall scores match. One decoder is trained with the brain input partly masked, so it is prior-heavy. The other is trained with counterfactual augmentation, so it is brain-heavy.
The third check is the granularity curve. We plot hard accuracy and accuracy on planted counterfactual scans against parcel count, from 17 networks to single voxels, with confidence intervals from held-out-subject resampling. We then compare this curve with the overall-accuracy curve. The core claim is supported if the two curves diverge, either because hard accuracy plateaus or falls at voxel level while overall accuracy keeps rising, or because hard accuracy keeps climbing. The claim is refuted if the two curves move together.
References
- Shifting the Baseline: Single Modality Performance on Visual Navigation & QA
- The Role of Linguistic Priors in Measuring Compositional Generalization of Vision-Language Models
- Chimera: Diagnosing Shortcut Learning in Visual-Language Understanding
- LIME: Less Is More for MLLM Evaluation
- MindLLM: A Subject-Agnostic and Versatile Model for fMRI-to-text Decoding
- BIT-LLM: Brain Instruction Tuned LLM with persistent Cross-Attention for fMRI-to-Text Decoding
- UMBRAE: Unified Multimodal Brain Decoding
- Neuro-Symbolic Decoding of Neural Activity
- Adapting Large Language Model for Cross-Subject Semantic Decoding from Video-Stimulated fMRI
- Robust Evaluation of Language-Brain Encoding Experiments
- Retinotopy and Functional Subdivision of Human Areas MT and MST
- Task vs. rest—different network configurations between the coactivation and the resting-state brain networks
- Autism: reduced connectivity between cortical areas involved in face expression, theory of mind, and the sense of self
- Методы и средства анализа сигналов головного мозга человека на данных функциональной магнитно-резонансной томографии (Methods and Tools for the Human Brain Signals Analysis over the Functional Magnetic Resonance Imaging Data)
- Inferring the temporal evolution of synaptic weights from dynamic functional connectivity
- A framework for studying synaptic plasticity with neural spike train data
- Model Based Inference of Synaptic Plasticity Rules
- The Algonauts Project 2023 Challenge: How the Human Brain Makes Sense of Natural Scenes
- Do 3D Large Language Models Really Understand 3D Spatial Relationships?