Splitting the Scan Budget: A Separation Principle for Shared and Subject-Specific Brain Encoding

Brain-imaging labs face a fixed budget of scanner hours and must choose between two designs. One scans a few people for many hours each, which is called deep phenotyping. The other scans many people briefly. Today that choice is made by habit. We propose to treat it as a resource-allocation problem, borrowing a lesson from wireless engineering. In multi-antenna relay networks, Fawaz et al. proved that the best transmission directions depend only on slow, long-run channel statistics, while the power on each direction can be tuned separately. That split makes an intractable joint problem easy. Our claim is that brain encoding models have the same structure. An encoding model predicts each voxel's fMRI signal from features of the stimulus, here the internal activations of a large language model. We conjecture that the brain-relevant subspace of those features is shared across people and can be learned from everyone's data pooled together. Each individual then needs only a small, cheap mapping from that subspace onto their own voxels.

If this split holds, two practical things follow. First, prediction error breaks into a term that depends on total pooled data and a term that depends only on each person's own scan time. That yields a closed-form rule for dividing a budget between number of participants and hours per participant. Second, the known plateau in language-model encoding performance should be a property of per-subject regression and not of the brain. Antonello et al. found that accuracy grows about 4.4% per tenfold increase in model size and then stalls near 30B parameters. They attributed the stall to too few samples for too many feature dimensions. Pooling across subjects to estimate the shared subspace attacks that bottleneck directly. It should move the plateau to larger models as the number of participants grows.

Core Claim

We model subject $i$'s voxel responses as $Y_i \approx X B W_i^\top$. Here $X$ holds the stimulus features over time (dimension $D$, the language model's width), $B$ is a $D \times K$ matrix spanning a shared brain-relevant feature subspace, and $W_i$ is a subject-specific $V_i \times K$ map onto that subject's $V_i$ voxels. The separation principle is the claim that a two-stage fit loses essentially nothing relative to fitting $B$ and all $W_i$ jointly. The first stage estimates $B$ from all $N$ subjects' data. The second stage fits each $W_i$ by ridge regression on $XB$. The condition is that subjects' true response maps share a column space, which plays the role that Kronecker-correlated channels play in the relay proof. Counting parameters against samples, as in standard reduced-rank regression variance arguments, gives an excess prediction error of roughly

$$\mathcal{E}(N,T,K) \approx a\,\frac{DK}{NT} + b\,\frac{K}{T} + \epsilon(K),$$

where $T$ is scan hours per subject, $a$ and $b$ are noise-dependent constants, and $\epsilon(K)$ is the error from truncating to $K$ shared dimensions. Under a budget $C = N(T + t_0)$, where $t_0$ is the fixed per-participant overhead for setup and anatomical scans, minimizing $\mathcal{E}$ gives the optimal split.

The form makes three falsifiable predictions. The model width $D$ is divided by pooled data $NT$, so the size at which bigger language models stop helping should grow with $N$. Per-subject cost scales with $K$ and not $D$. A new participant, for example a patient in a small clinical cohort, should therefore reach useful accuracy with far shorter scans once $B$ is fixed. Finally, the best design may be mixed: a few long-scanned "anchor" subjects to pin down $B$, and many short-scanned subjects. This is a discrete choice layered on the continuous choice of hours. It mirrors how Hajiaghayi et al. separate discrete channel pairing from continuous power in relay networks.

Relation to Prior Work

Three lines of neuroimaging work already split shared from individual structure, and all of them supply parts of our method. Chen et al.'s shared response model factors each subject's data into a shared time course $S$ and an orthonormal subject basis $W_i$, and shows that low rank denoises. Because $S$ is indexed by time, it requires every subject to experience identical, time-locked stimuli. Our shared object lives in stimulus-feature space instead, so subjects can hear different stories. Khosla et al. train a shared convolutional backbone with subject-specific heads on 10 HCP 7T subjects and beat per-subject models. Their cohort is fixed, and they offer neither a theory of when sharing is lossless nor any budget analysis. Antonello et al. supply the scaling facts that motivate us, along with their pipeline: Lanczos interpolation, delayed-response features, and ridge regression on the Moth Radio Hour data. They fit each subject independently and vary only $T$ within $N=3$.

The wireless papers are a lens, not a source of transferable math. Fawaz et al. show that directions set by long-run statistics separate from per-path power. Hajiaghayi et al. show that the discrete part of a joint problem can dominate the gains; pairing with uniform power comes close to the joint optimum. Huang and Ansari show that a structural bound shrinks the combinatorial search. These papers also constrain us. Their separations are exact only under idealized assumptions, such as noise-free relays and orthogonal channels, and break when those fail. We therefore treat our separation as a hypothesis whose loss must be measured, not assumed.

Among the motivating small-cohort papers, Cao et al. and Shiell et al. document that high cost limits cohort size and that small cohorts fail to replicate. That is the setting where a cheap per-subject stage would matter. Gazula et al. pool nearly 2,000 subjects through federated summary statistics, but they treat site data as fixed and do no allocation.

Research Program

We will build the two-stage estimator. The shared subspace $B$ is found by reduced-rank ridge regression on stacked subject data. Voxel counts differ across subjects, so each subject's responses are first projected onto a per-subject response basis. $K$ is chosen by pooled cross-validation. We will then fit a joint alternating-minimization version of the same model, so the separation loss can be measured directly as the accuracy gap between the joint and two-stage fits.

The analysis has two parts. We will derive $\mathcal{E}(N,T,K)$ more carefully, including an explicit ridge penalty, and state the condition on between-subject subspace misalignment under which the two-stage fit matches the joint fit. We will also solve the budget problem, both with equal hours per subject and with the anchor-plus-shallow design.

The empirical part fits $a$, $b$, and $\epsilon(K)$ from a small corner of the $(N,T)$ grid and asks whether the law predicts accuracy everywhere else. Two datasets cover the two axes. The Moth Radio Hour data from Antonello et al. cover the hours axis, since $T$ can be subsampled up to 20 hours, but they have only three subjects. The HCP 7T movie-watching protocol used by Khosla et al. covers the participant axis, with many subjects and short scans. On HCP we use language-model features of the movie soundtrack and dialogue together with visual features. Sherlock and Forrest from Chen et al. serve as time-locked sets for comparing against the shared response model.

Baselines

The naive reference is the per-subject voxelwise ridge regression of Antonello et al., run on the same features, the same train/test story splits, and the same ridge grid. It is the baseline behind the 30B plateau. The strongest reported figures come from that paper: roughly a 15% gain in held-out correlation from 125M to 30B parameters, 4.4% per tenfold increase in model size, and 122% per tenfold increase in fMRI data. Beating it means one of two things. Either the two-stage model's accuracy keeps rising past 30B parameters when $N$ is large enough, or it matches per-subject ridge on a new subject using a fraction of that subject's hours.

The strongest existing multi-subject method is the shared backbone with subject heads of Khosla et al.. For a fair comparison we give it the same frozen language-model features in place of its CNN extractor, the same subjects, and a matched hyperparameter search budget. The file reports no numeric result for it on Lacuna; it states only that it "consistently outperformed individual linear and CNN-based models". We count as beating it matching its accuracy while also predicting its accuracy under reallocated budgets, since it offers no such prediction.

On the time-locked datasets we add SRM followed by ridge (Chen et al.). Its reported audiobook gain, classification accuracy rising from 0.72 to 0.82, measures a different task, so we compare on encoding correlation.

Two ablations isolate the key ingredient, which is that $B$ is learned from pooled brain data. The joint-fit ablation tests whether separating the stages costs accuracy. The stimulus-only ablation replaces $B$ with the top-$K$ principal components of $X$ itself, computed from no brain data.

Why not just…?

Why not just reduce the language-model features with PCA before per-subject ridge? This is the most serious threat to the claim about the plateau. Stimulus PCA also cuts $D$ to $K$ and may by itself fix overfitting at 30B+. PCA keeps directions of high feature variance, not directions that predict brain activity, and nothing in the file shows those coincide. The stimulus-only ablation settles this directly. If PCA-reduced ridge matches the pooled subspace at every $N$, the cross-subject part of our claim is wrong and only the budget law survives.

Why not just fit an empirical power law to accuracy over a grid of subject counts and hours? A purely empirical fit is honest and cheap, and on a dense grid it may predict as well as our law. Its weakness is extrapolation. It has no reason to predict that the plateau moves with $N$, or that mixed anchor designs beat uniform ones. We will fit both on the same small grid corner and compare their errors on held-out allocations. If the empirical law wins there, the theory adds nothing.

Why not just scan everyone longer, given that tenfold more data raised accuracy by 122%? More hours is the budget itself, not an alternative to allocating it. The same finding shows returns are logarithmic, so the marginal hour is worth less on a subject who already has many. That is exactly why reallocating hours across subjects could pay off.

Why not just train Khosla's shared network end-to-end? It may match our accuracy. Our aim is not higher accuracy at fixed $N$ and $T$. It is a stated condition under which sharing is lossless, plus a budget rule. If the shared network clearly outperforms our linear model, we will test whether its gains also follow the $NT$ and $T$ decomposition.

Decisive Checks

The first check is separation loss. The two-stage fit should lie within the across-split noise of the joint fit on both datasets. If it falls clearly below, the core claim fails.

The second check is plateau migration. On HCP 7T we fit encoding models across language-model sizes for increasing $N$. The law predicts that the peak model size grows with $N$. A peak fixed near 30B regardless of $N$ would refute it.

The third check is transfer to new subjects. We fix $B$ using the three deep Moth subjects, or a larger HCP subset, and fit held-out subjects on 5 to 60 minutes each. We then compare against per-subject ridge on the full data, which is the regime relevant to small clinical cohorts.

The fourth check is allocation prediction. For three total budgets we compare the law's recommended split, both uniform and anchor-plus-shallow, with the empirically best split on the grid. Success means the recommended design falls within noise of the best one, where the empirical power law does not.