What Is a Connectome Worth in Training Runs? Pricing a Cortical Prior for Architecture Search

Architecture search is expensive because almost every way of judging a candidate network involves training it. Early methods used thousands of GPU days, and the energy spent searching can exceed what the resulting model ever saves in deployment (survey on low-power deep learning). Biology offers something no other prior does: a description of good wiring that costs zero training runs. Graph Structure of Neural Networks found that network accuracy varies smoothly with two wiring statistics. The first is clustering, which measures how often a node's neighbours are also connected to each other. The second is average path length, the typical number of hops between two nodes. The best networks sat in the same region these statistics occupy in the macaque visual cortex.

This proposal asks a quantitative question that nobody has answered. Suppose we restrict a search space to the region measured in real connectomes, without looking at any trained network. How many trained-model evaluations is that restriction worth? Does that value carry over to new architecture families and datasets, where architecture rankings are known to break down? The idea fits in two sentences. Treat connectome statistics as a free, pre-registered filter on which wirings to consider, then measure how many training runs a data-driven filter needs to do equally well. If that number is large and stable across tasks, biology is a useful compute-saving prior. If it is a handful of three-epoch runs, the biological story is decorative, and we will say so.

Core Claim

We claim that a region of wiring-statistic space fixed in advance from connectome measurements does three things, at zero search cost. First, it raises the quality of a randomly drawn architecture enough to cut the number of evaluations random search needs to reach near-optimal accuracy. Second, it transfers across datasets better than transferring individual top architectures does. Third, it is worth more than the cheapest data-driven alternative of equal coverage. The contribution is the measurement, not a new search algorithm. We define a "price" for the prior: the number of short training runs a data-fitted region needs before it matches the prior's search efficiency. The paper reports that number across scales and tasks.

This differs from the closest work in specific ways. You et al. found the sweet spot by training thousands of graphs. They noted the match to macaque cortex afterwards and never used the cortex to save compute. They also searched only channel-level wiring inside standard stages, with no comparison against modern training-free scoring. NASGraph scores individual architectures by generic graph properties such as average degree and wedge counts, which are cheap, training-free proxies for accuracy. It applies no biological target and does not restrict the space. MSE-NAS builds cortical motifs into evolutionary search with a training-free firing-pattern score, but only for spiking networks and never on standard benchmarks. CogNAS uses brain-activity fit as the search goal, which adds cost rather than removing it. BioNAS uses sequence motifs, not wiring, inside an expensive reinforcement-learning search.

Avoiding Circularity

There is an obvious objection. The macaque region came from You et al.'s own artificial-network sweep, so using it might just reuse their result. We handle this in two ways. First, we pre-register the prior regions from connectome data alone, before running any experiment: macaque cortex statistics, plus statistics computed from the Drosophila connectome (Zhang et al., 2025) and C. elegans (Bardozzo et al., 2024). Second, we test where You et al. did not search: macro-level sparse directed acyclic graphs (DAGs), meaning layer-free networks in which any neuron may feed any later neuron, and convolutional cells at new scales and datasets.

We also impose a hardware constraint on every prior region. Each region also bounds topological height, the number of sequential layers after 4Ward's longest-path layering. This keeps wall-clock cost honest. Both RegNet and You et al. warn that fewer FLOPs do not mean faster GPUs.

Research Program

We build three search spaces where clustering and path length mean something. This rules out NAS-Bench-201 cells as the main testbed, because a four-node, six-edge cell has nearly undefined clustering.

Space A follows You et al.'s relational graphs. It covers a 5-layer MLP on CIFAR-10 and a ResNet-style CNN on CIFAR-100 and ImageNet-16-120, with graphs drawn from the WS-flex generator, a generalised small-world graph sampler that varies clustering and path length independently. Space B contains sparse neuron-level DAGs of 256 to 1024 units at fixed edge budgets, trained through 4Ward's masking. Space C is the NDS DARTS space, where we compute statistics on the unrolled whole-network graph and on NASGraph's channel-activation graph. Space C links our results directly to existing proxy benchmarks.

For Spaces A and B we fully train a stratified sample of several thousand graphs over three seeds. That gives a small tabular benchmark which we release. Training is cheap at these scales, and we also log 3-epoch accuracies, which lets us compare against the short-training proxy.

On each space we compare three kinds of prior region of equal area in statistic space: the biological regions, placebo regions placed at random, and data-fitted regions. Data-fitted regions are obtained in the RegNet style, by fitting error distribution curves to $n$ short training runs on a source dataset. The price of the biological prior is the smallest $n$ at which the data-fitted region matches the biological one in samples-to-target. Samples-to-target is the expected number of fully evaluated draws needed to come within 0.5% of the space's best accuracy.

We also run a transfer test. Each region is fit or fixed on CIFAR-10 and then used unchanged on CIFAR-100 and ImageNet-16-120. We compare this with transferring the top 1% of individual architectures, which NATS-Bench reports transfers poorly.

Baselines

The naive reference is uniform random search over the full space, with FLOPs and parameters held fixed. NAS-Bench-101 shows this is hard to beat, because most of that space already trains well (NAS-Bench-101). Beating it means a statistically significant reduction in samples-to-target across seeds, on at least two of the three spaces.

We also run regularized evolution from the same benchmark line. It gets the same number of evaluations, and it gets the prior region as an optional constraint, so we can see whether the prior helps a stronger optimizer as well as random search.

The strongest training-free competitors are proxy-filtered random search using SynFlow, NASWOT, EZNAS-A, and NASGraph's average-degree-plus-Jacobian-covariance score. The best of these reported in the file is EZNAS-A, with Kendall's tau 0.65 on NAS-Bench-201 CIFAR-10 against 0.57 for NASWOT and SynFlow (EZNAS). NASGraph reports a 12–14% gain in Spearman correlation from its combined score. We report the proxies' rank correlation on our spaces, both overall and within the top decile. More importantly, we report samples-to-target when each proxy prunes the space to the same fraction the biological region keeps.

The strongest training-based estimator is the 3-epoch proxy, which You et al. report reaches a 0.93 Pearson correlation with 100-epoch ImageNet accuracy. We charge it its true GPU cost in the same accounting. The RegNet-style data-fitted region is the most direct competitor, and it defines the price.

The ablations isolate what matters. We compare macaque statistics against generic small-worldness (any graph more clustered than random at similar path length), against other connectomes, and against placebo regions. We also remove the topological-height bound to measure its latency effect, with latency timed on real GPUs.

Why not just…?

Why not just use random search, since most architectures are fine anyway? If the space is nearly flat, no prior can help, and our samples-to-target numbers would show that directly. You et al.'s 1.4% CIFAR-10 gap for sweet-spot graphs over complete graphs at equal FLOPs suggests the relational-graph space is not flat. Whether random DAGs are flat is an open question that Space B answers.

Why not just filter with an existing zero-cost proxy like NASGraph or EZNAS-A? These proxies score individuals and are known to lose resolution in the top decile, as the EZNAS authors acknowledge. A region prior instead shapes the population being sampled. The two are complementary, and we test whether proxy filtering inside the biological region beats either one alone. If NASGraph's score alone matches the region on samples-to-target, the biological contribution reduces to "use graph statistics," and we will report it that way.

Why not just fit the good region from a few cheap runs, RegNet-style? This is the real threat, and the whole design is built around it. If roughly a dozen 3-epoch runs on CIFAR-10 recover a region that is just as good and transfers just as well, the connectome is worth almost nothing. The price metric reports exactly this. The biological prior wins only if its price is high or if it transfers better than fitted regions do.

Why not let dynamic sparse training discover the wiring during training? Zhang et al. (2025) show that rewiring during training can beat dense MLPs at 1% connectivity with no separate search. We run their soft-sampling variant with sigmoid density decay at matched density on Space B as a baseline. We also measure where its learned graphs land in clustering and path-length space. If they drift into the cortical region, that supports the prior. If they reach the same accuracy from elsewhere, the region is not necessary.

Decisive Checks

The proposal succeeds if three things hold. The pre-registered macaque region beats equal-area placebo regions on samples-to-target in at least two spaces. Its price exceeds the cost of the 3-epoch-fitted region with the same transfer performance. It retains its advantage when moved from CIFAR-10 to ImageNet-16-120 better than top-1% architecture transfer does, and it does so without increasing measured GPU latency under the height bound.

The proposal fails cleanly if placebo regions or generic small-worldness perform as well, or if the price is a handful of cheap runs. Either outcome is worth publishing, because it settles whether brain wiring saves search compute or only looks good in hindsight.