Step-Level Batching for Adaptive Cortical Surface Flows

Recent deep learning models rebuild the folded surface of the human cortex from an MRI scan in about a second. They do it by solving an ordinary differential equation (ODE): they push a template mesh along a learned velocity field until it fits the brain. Every published model runs this solver for the same fixed number of steps on every brain. The usual reason is practical: if subjects are batched together on a GPU, a per-brain step count would force the whole batch to wait for the slowest brain. The fixed budget is a compromise. It wastes steps on easy, young, well-imaged brains, and it can be too coarse for atrophied brains with deep, narrow sulci. Those deep folds are exactly where the discrete solvers still produce self-intersecting faces, as CorticalFlow and Vox2Cortex-Flow both acknowledge.

We propose to remove the compromise. Each brain gets its own error-controlled step count. A serving engine schedules work one integration step at a time, so that when one brain finishes, a newly arrived brain immediately takes its GPU slot. This is the same idea as continuous depth batching for looped language models (Depth-adaptive Inference of Looped Language Models via Continuous Depth Batching), transplanted to geometric mesh flows. The claim is that adaptivity and batching together give a better accuracy-versus-throughput curve than either alone. Concretely, at matched GPU throughput, hard brains get fewer self-intersections and lower surface error. Or, at matched accuracy, a shared GPU processes substantially more subjects per hour.

Core Claim and What Is New

No paper in our search connects continuous or iteration-level batching to neuroimaging. The obvious version of that connection is a GPU inference service for brain MRI, in the style of SONIC. That system is described in GPU-accelerated machine learning inference as a service for computing in neutrino experiments: legacy CPU physics jobs send 48×48 detector patches over the network to an Nvidia Triton server, which pools them into batches. It reports a 17x speedup of the machine learning module and finds that one GPU serves 68 CPU threads. Porting this architecture to MRI is mostly engineering. For models with fixed step counts, ordinary request-level batching in Triton already captures most of the benefit, because every request costs the same.

The idea becomes non-trivial only once per-subject cost varies, and in current cortical reconstruction models it does not vary. We therefore make it vary, deliberately and for a reason: to spend compute where the anatomy needs it. Two ingredients follow from that choice.

The first ingredient is a per-brain stopping rule for the mesh ODE. We use an embedded Runge–Kutta error estimate, meaning two solver formulas of different order computed together so their disagreement estimates the step's error. The estimate is aggregated over the vertices of each surface. A cheap self-intersection count on the coarse mesh serves as a secondary check.

The second ingredient is a scheduler that treats each (brain, surface) pair as a request that occupies a slot for a variable number of steps. The scheduler separates two stages. The heavy 3D convolutional encoder runs once per scan, much like the "prefill" stage of a language model, which processes the prompt once. The light, repeated mesh-deformation steps play the role of "decode". BATON separates prefill from decode for language models in the same way.

Write $K_i$ for the number of solver steps brain $i$ needs and $K_{\max}$ for the fixed budget a static batch must run. The ideal throughput gain of step-level refill over a static batch run to $K_{\max}$ is then

$$S = \frac{K_{\max}}{\mathbb{E}[K_i]},$$

because a static batch pays $K_{\max}$ step-slots per brain while perfect refill pays only $K_i$. This is the same notion of "theoretical maximum speedup" that continuous depth batching was measured against: it achieved 94–99% of that bound on language models.

The whole proposal therefore depends on an empirical question nobody has answered. How wide is the distribution of $K_i$ across real cohorts, and does giving the tail more steps actually fix the defects in hard brains?

Research Program

We build on the two published models whose deformation is an explicit numerical integration. Vox2Cortex-Flow integrates a graph neural ODE, with edges linking the inner (white matter) and outer (pial) surfaces; it runs in about 1.6 s with surface error below 0.185 mm relative to FreeSurfer. CorticalFlow is a multi-scale cascade of diffeomorphic deformation modules growing from 30k to 435k vertices, running in about 1.1 s for four surfaces. We will use released code if it exists and otherwise reimplement and retrain on ADNI. The stationary-velocity-field models (Coupled Reconstruction of Cortical Surfaces by Diffeomorphic Mesh Deformation, SurfNN) serve as secondary targets, because their integration step count can also be made adaptive.

The work has three phases.

Phase 1: measurement. We sweep step counts per subject on ADNI, OASIS-1 and the MGH clinical scans used by End-to-End Cortical Surface Reconstruction from Clinical Magnetic Resonance Images. For each brain we record the smallest step count at which surface error, self-intersection rate and cortical thickness stop improving. This produces the empirical distribution of $K_i$ and its correlation with age, atrophy and slice thickness.

The vector fields were trained with fixed steps, and the network may have learned to exploit that particular discretization. We therefore test two settings. In the first, we integrate the trained fields adaptively without retraining. In the second, we fine-tune with randomized step counts so that the learned field better approximates a true continuous flow.

Phase 2: the engine. The engine separates the work into an encoder pool and a flow pool. Each in-flight brain occupies three things: its encoder feature volume, which the mesh vertices sample from; its mesh state; and its solver state. When a request's error estimate falls below tolerance, the request exits and its slot is refilled at the next step boundary.

We expect in-flight concurrency to be limited by GPU memory for the feature volumes, not by compute, because a 256³ volume is large (GPU-accelerated machine learning inference as a service for computing in neutrino experiments and the file's own memory analysis both point to batch sizes of one or two during the encoder stage). We will therefore study two ways to shrink that footprint: storing the feature volumes in reduced precision, and cropping them to the mesh's bounding region.

For the multi-scale cascade in CorticalFlow, a brain's stage boundary (coarse to fine mesh) changes its per-step cost. We handle this with separate queues per resolution, analogous to the prelude, core and coda queues in continuous depth batching. That paper flags these boundary stages as its own bottleneck, and we measure whether the same happens here.

Phase 3: queueing under realistic arrivals. Scans arrive at random times in hospitals and in consortium-scale processing. Following A Queueing Theoretic Perspective on Low-Latency LLM Inference with Variable Token Length, we model each scan's service time as a draw from the measured step distribution. We then compare the predicted delay with measured delay under Poisson arrivals, including a tail cap that limits steps at a fixed percentile.

The key difference from that work concerns what a capped request loses. In language modeling, clipping the longest requests truncates their output. Here, capping steps on the hardest brains degrades exactly the surfaces that most need care. So we report the quality cost of the cap explicitly rather than treating it as a free reduction in delay.

Baselines

The naive reference point is how these tools are run today: one process per subject, batch size one, fixed step count, sequential on one GPU. We measure it with the model already loaded, which favors the baseline, and separately including process start-up, which reflects command-line practice. Beating it is expected and is not the claim.

The strongest existing competitor is SONIC-style request-level dynamic batching (GPU-accelerated machine learning inference as a service for computing in neutrino experiments). Here the full fixed-step model is served from Triton, which groups whole-subject requests into batches up to a memory-limited size. It is the strongest competitor because, with fixed steps, every request costs the same and request-level batching is close to optimal. Any gain we show must come from adaptivity plus refill, not from serving infrastructure.

We run it on the same GPU and the same arrival traces, with its batch size and queue delay tuned on a held-out trace using the same tuning budget as our scheduler. It runs at two fixed step budgets: the published default, and the larger budget needed for its worst-case brains to reach the quality our adaptive method achieves.

The bar for success is either of two outcomes. The first is at least 1.3x more subjects per GPU-hour at matched mean surface error and matched self-intersection rate on the hardest decile of brains. The second is a significant reduction in self-intersections on that decile at matched throughput.

No reported result found on Lacuna for multi-subject serving throughput of any cortical reconstruction model. The only per-model figures available are single-subject latencies, such as about 1.1 s for CorticalFlow and about 1.6 s for Vox2Cortex-Flow. The closest analogous figure for our mechanism is 1.5–1.9x offline throughput over standard continuous batching, reported for continuous depth batching on looped language models, with GPU idle time falling from 40% to 0.67%.

Three further baselines and ablations isolate the key ingredient. The first, "adaptive steps with static batching", lets each brain stop early but keeps its slot idle until the batch finishes. It isolates the value of refill. The second, "refill with fixed steps", should give essentially no gain, because no request finishes early. It confirms that adaptivity is what creates the opportunity. The third, "monolithic versus decoupled encoder and flow", measures the benefit of separating the stages, analogous to the prefill/decode split in BATON.

Why not just…?

Why not just batch whole subjects in Triton with fixed steps, since each brain already takes a second? If Phase 1 finds that the step distribution is narrow, this alternative wins outright, and we will report that. A narrow distribution would be an informative negative result: it would mean the fixed-step design is not leaving much on the table. It is the main threat to the proposal, which is why Phase 1 comes first and why the Triton baseline is the reference throughout.

The proposal survives only if the hardest brains need several times the median step count. The SONIC result shows that pooling requests from many clients onto one GPU pays off when per-request costs are uniform; it does not tell us what happens when they are not.

Why not just raise the fixed step count for everyone? This fixes the hard brains at a cost proportional to the ratio $K_{\max}/\mathbb{E}[K_i]$, paid on every subject. It appears directly in our experiments as the "matched quality" fixed-step baseline. If its throughput loss is small, the adaptive method is unnecessary. The measured step distribution determines the answer in advance.

Why not just sort scans by predicted difficulty and run static batches of similar brains? This is the length-bucketing trick from language-model serving, and it is a real competitor. It needs a predictor of each brain's step count from the scan before the flow runs, and it delays scans while buckets fill. We implement it with the best predictor we can train from encoder features, giving it oracle step counts as an upper bound. We compare its tail latency under Poisson arrivals with step-level refill. If it matches refill when given oracle step counts, refill's advantage is only that it needs no such predictor, and we will say so.

Decisive Experiments

The first decisive check is the Phase 1 histogram of step counts on ADNI, OASIS-1 and MGH clinical scans. Alongside it, we test whether extra steps on the tail reduce the percentage of self-intersecting faces and the cortical thickness error. The clinical reference point is 0.006% self-intersections and 0.24 mm thickness error, reported in End-to-End Cortical Surface Reconstruction from Clinical Magnetic Resonance Images. If the ratio $K_{\max}/\mathbb{E}[K_i]$ is below about 1.3, or if the tail shows no quality gain, we stop the throughput work and publish the measurement.

The second check is reproducibility. Adaptive stopping must not make surfaces less consistent across repeated scans of the same subject. We use the test-retest data that Vox2Cortex and Vox2Cortex-Flow used to show better consistency than FreeSurfer, and require that vertex-wise consistency be no worse than under fixed steps.

The third check is a sweep of offered load under Poisson arrivals on one GPU. For each scheduler we measure subjects per GPU-hour, 95th-percentile latency, GPU idle fraction and quality on the hardest decile. We also report the fraction of the bound $S$ that refill actually achieves, the same measure on which continuous depth batching reached 94–99% for language models.