Deep networks can now predict a person's task-evoked brain activity from a short resting scan. A resting scan records spontaneous brain activity while the subject does nothing in particular, and the predicted output is the activation map that a language or social-reasoning task would have produced in that person. These predicted maps look sharper and more reliable than the person's own measured maps. That is the problem this note addresses. In accelerated MRI reconstruction, a generator asked to fill in data it never measured falls back on what it learned from the training population and produces plausible structure that is not there. A rest-to-task predictor faces the same situation. Where the subject's resting data say little, the network can output a crisp, typical-looking activation pattern drawn from the group. The result appears individualized but is really the population average. When the subject's data are noisy, the network can instead invent sharp "individual" features that would not replicate.
We propose to measure and expose this failure directly. For every voxel of a predicted individual map, we ask two questions. Does this voxel's departure from the group average actually depend on the subject's own data? Does it replicate when the subject's resting data are resampled? The resulting per-voxel "support map" requires no ground-truth task scan at test time. It turns a single predicted map into a map plus an honest statement of which individual features the data back.
Core Claim
Current rest-to-task predictors are scored on whole-map correlation with a measured task map, and this score can be dominated by the shared group pattern. Predicting Task fMRI Contrasts from Resting-State fMRI Using Sparse 3D Convolutions (BrainSparseCNN, a sparse 3D U-Net mapping voxel-wise resting connectomes to 47 Human Connectome Project task contrasts) reports the relevant numbers. Its average cross-task Pearson correlation is 0.672. The plain group average reaches 0.594, and the subject's own repeat scan reaches only 0.611. A predictor that beats the person's own retest is either removing noise or substituting the group prior for the individual, and whole-map correlation cannot tell these apart. Rest2Visual is a conditional decoder that predicts visually evoked maps from a single-seed resting map. It openly reports predicted maps with higher signal-to-noise than the ground truth, which raises the same ambiguity.
Our claim is that the meaningful quantity is the subject-specific deviation from the group. A predictor can fail on it in two ways that mirror anatomical hallucination. The first is erasure: true individual features are replaced by the group pattern. The second is fabrication: confident individual-looking features appear that the data do not support. Both can be detected without test-time ground truth by perturbing the model's input.
Let $f$ be a trained predictor, $x$ the subject's resting connectome, and $x_0$ a null input carrying no subject information, namely the group-mean connectome. The data-driven part of the prediction is $d = f(x) - f(x_0)$. Let $x^{(b)}$ be connectomes recomputed from bootstrap resamples of the subject's resting runs. The per-voxel support score is
$$ s_v = \frac{\operatorname{mean}_b \, d_v^{(b)}}{\operatorname{sd}_b \, d_v^{(b)}}, $$
where $d_v^{(b)} = f(x^{(b)})_v - f(x_0)_v$ at voxel $v$. A score near zero with low spread marks prior-filled voxels, where the output ignores the subject. A large spread marks fabrication-prone voxels, where the output tracks noise in the subject's data. Gating the deviation by $s_v$ gives a conservative map that only claims individuality where the data support it.
Relation to Prior Work
The work that motivates this proposal comes from generative medical imaging. Preserving Structural Fidelity in Generative Medical Imaging describes how unconstrained generators fill the unmeasured part of an ill-posed problem from their training prior. Rethinking Perceptual Metrics for Medical Image Translation shows that standard image-quality scores can be near-uncorrelated with anatomical correctness. We carry both lessons into functional prediction: the inverse problem here is resting data to task map, and the misleading metric is whole-map correlation.
Other work constrains the design. Anatomical guardrails such as segmentation conditioning (SegGuidedDiff), template deformation (Diff-Def), structural losses (AG-LDM), and persistence-diagram topology losses (TPOT) all keep outputs anatomically valid. Functional erasure, however, happens inside anatomically valid gray matter. Functional borders also separate from folding patterns, as The Quest for the FFA and Where It Led argues. These anatomical tools therefore address a different failure, and we do not reuse them.
Two sources supply methods. The first is test-time risk estimation without ground truth, as in SMRD, which embeds Stein's unbiased risk estimate in diffusion sampling for MRI. The second is the split-data discipline of functional localizers (Cortical Face-Selective Responses Emerge Early in Human Infancy), where voxels are defined on one data partition and measured on another. Our support score combines input perturbation with resampling in that spirit. It does not modify the predictor, so it applies to any rest-to-task model.
The same question applies to individualized parcellation. Individual-Specific Areal-Level Parcellations Improve Functional Connectivity Prediction of Behavior (MS-HBM) is a hierarchical Bayesian model initialized from a group atlas. It reports that parcellations from 10 minutes of resting data generalize better than competitors using 150 minutes. A strong prior is exactly what makes such data efficiency possible. Whether the 10-minute individual borders are data-driven or prior-filled has not been separated in the file's material, and this is our secondary target.
Research Program
The first component is an evaluation protocol centered on deviation. On HCP test subjects we compute the group mean $m$ for each contrast. We then score predictions by the correlation between $f(x)-m$ and $y-m$, where $y$ is the measured map. Using the two HCP phase-encoding runs of each task, we compute a noise ceiling for that deviation correlation. This tells us how much of each published model's 0.672 survives once the shared pattern is removed.
The second component is the support map defined above. We compute it for BrainSparseCNN and for a surface-based predictor trained under an identical protocol. We validate it against held-out task data in two ways. The first test asks whether high-support deviations match the sign of the measured deviation in an independent task run. The second asks whether low-support voxels with large measured deviations, our erasure cases, are where the predictor collapses to $m$.
The third component is a gated predictor that emits $f(x_0) + \mathbb{1}[s_v>\tau]\,d$. We tune the threshold $\tau$ on validation subjects for deviation fidelity, not whole-map correlation.
The fourth component applies the same audit to MS-HBM on the Midnight Scanning Club. This cohort has 10 subjects with 300 minutes each. In this setting the 10-minute parcellation is the prediction and the 300-minute parcellation is near-truth. The null input is the group atlas initialization, and bootstrap resamples are drawn across sessions.
Baselines
The naive reference is the group-average map, which BrainSparseCNN reports at 0.594 cross-task correlation. Under our deviation metric this baseline scores zero by construction, which makes it a useful floor. The empirical retest map is the second reference: 0.611 correlation and 89.7% subject identification as reported in the BrainSparseCNN paper. It is the only reference containing no learned prior.
The strongest existing predictor is BrainSparseCNN, reported at 0.672 cross-task correlation, 98.3% subject identification, and SOCIAL theory-of-mind Dice of 0.169–0.188. We retrain it with the released configuration on the same 919/39 HCP split, using the same compute and validation budget as every other model. BrainSurfCNN, reported at 0.620, is the second learned baseline. Both are evaluated on whole-map correlation and on deviation correlation.
Our support map is a flagging method, so its true competitors are other uncertainty estimators. These are a five-member deep ensemble of BrainSparseCNN and Monte Carlo dropout, each given the same training budget. Each estimator produces a per-voxel score, and we compare them on how well that score predicts deviation errors on held-out task runs. Beating these competitors means a clearly higher area under the curve for detecting both erasure and fabrication voxels. It also means the gated predictor gives higher deviation correlation than ensemble-variance gating at matched coverage.
Two ablations isolate the key ingredients. The first removes the null-input term and keeps only bootstrap variance. The second removes the bootstrap and keeps only the null-input difference. If either alone matches the full score, the proposal reduces to that simpler piece, and we will report it that way.
Why not just…?
Why not just use ensemble or dropout uncertainty? Ensembles measure disagreement between models. When every member has learned the same population prior, they agree confidently in prior-filled regions, so erasure should look certain. This is a hypothesis, not a cited result, and the matched comparison above settles it. If ensemble variance detects erasure as well as our score, our main contribution narrows to the deviation-centered evaluation protocol.
Why not just report correlation on group-subtracted maps and stop? This is a real alternative, and we adopt it as our primary metric. It only works where a measured task map exists, though. Clinical uses, such as predicting language dominance from about 10 minutes of rest as in Individual-specific resting-state networks predict language dominance in drug-resistant epilepsy, have no task map at test time. A per-voxel flag that needs no ground truth is what the metric alone cannot supply.
Why not just add anatomical or topological constraints, as hallucination work does for structural MRI? Those constraints keep outputs inside plausible anatomy. A predictor that pastes the group language network onto a right-lateralized subject violates no anatomical or topological rule. The mirroring experiment below directly tests this failure, which such constraints cannot catch.
Why not just scan longer? Long acquisitions are the right ground truth, and we use the 300-minute MSC data exactly that way. They are not viable for the patient groups that motivate prediction from short scans.
Proposed Experiments
The first decisive check is a lateralization flip. We mirror test subjects' resting connectomes left-to-right and also mirror their measured task maps, which creates subjects whose true language and theory-of-mind maps are atypically lateralized. A predictor driven by the subject's data should follow the flip. A prior-filled predictor should keep the typical left-dominant pattern. We measure what fraction of the language-contrast laterality index each model recovers. The support map should be low exactly where a model fails to follow the flip. Mirroring is imperfect because of real hemispheric asymmetries, so we also stratify the unmodified HCP subjects by the measured laterality index. We then report deviation fidelity on the most atypical tenth.
The second check is the deviation decomposition of the published numbers. If BrainSparseCNN's deviation correlation is close to its split-half noise ceiling, prior filling is minor. In that case the support map becomes a calibration tool rather than an alarm, and we will say so.
The third check is dose-response. We truncate the resting input from the full HCP amount down to 5 minutes. Support should fall, and deviation fidelity of the ungated model should fall faster than its whole-map correlation. Such a gap would be direct evidence that whole-map correlation hides the substitution of prior for data.
The fourth check is the MSC parcellation audit. For each cortical vertex we measure how often the 10-minute MS-HBM label disagrees with the group initialization. We then measure how often those disagreements agree with the 300-minute parcellation. Low-support disagreements should be wrong far more often than high-support ones.
All functional fidelity claims will be made on deviation metrics with noise ceilings. Perceptual-style scores will not be used, following the evidence in Rethinking Perceptual Metrics for Medical Image Translation.
References
- Predicting Task fMRI Contrasts from Resting-State fMRI Using Sparse 3D Convolutions
- Rest2Visual: Predicting Visually Evoked fMRI from Resting-State Scans
- Fast MRI Reconstruction: How Powerful Transformers Are?
- Rethinking Perceptual Metrics for Medical Image Translation
- Anatomically-Controllable Medical Image Generation with Segmentation-Guided Diffusion Models
- Diff-Def: Diffusion-Generated Deformation Fields for Conditional Atlases
- Anatomically Guided Latent Diffusion for Brain MRI Progression Modeling
- TPOT: Topology Preserving Optimal Transport in Retinal Fundus Image Enhancement
- The Quest for the FFA and Where It Led
- SMRD: SURE-Based Robust MRI Reconstruction with Diffusion Models
- Cortical Face-Selective Responses Emerge Early in Human Infancy
- Individual-Specific Areal-Level Parcellations Improve Functional Connectivity Prediction of Behavior
- Individual-specific resting-state networks predict language dominance in drug-resistant epilepsy