Brain-Template Geometry as a Free Lunch for Encrypted 3D MRI Inference

Homomorphic encryption lets a cloud server run a neural network on a patient's scan without ever seeing the scan. The server computes directly on encrypted numbers and returns an encrypted answer that only the hospital can read. The cost is steep. Encrypted arithmetic is orders of magnitude slower than ordinary arithmetic, ciphertexts and their keys run to gigabytes, and the best engineering so far has targeted small 2D photographs. A structural brain MRI is a dense 3D grid of about five million voxels (160×192×160 in the UK Biobank models of Accurate brain age prediction with lightweight deep neural networks). Nobody in the literature we found has run an encrypted deep network on such a volume.

This note argues that neuroimaging's standardization culture contains a resource that encryption systems have ignored. Once scans are put into a shared template space, a detail that standardized layouts such as the Brain Imaging Data Structure record in machine-readable form, every subject's volume has the same grid. It also has the same public brain mask, the template's outline of where brain tissue can be. Large parts of the box are known in advance to be background. In encryption, anything known to both parties can be compiled into the computation for free, while anything secret must be paid for. We propose to compile the template mask and the shared grid into the layout of the encrypted data, and so into the network's computation plan. Encrypted work is then spent only on 3D tiles that can contain brain, and many subjects are packed into each ciphertext. We test whether this makes encrypted 3D brain-age prediction practical at accuracy close to plaintext.

Core Claim

The claim has a concrete form. The cost of encrypted 3D convolution can be predicted from three quantities that are fixed by the public template, not by the patient: the number of active tiles at each layer, the halo width those tiles need, and the batch size the shared grid allows. Here a tile is a small 3D block of voxels that is packed into one stretch of a ciphertext, and the halo is the border of neighbouring voxels a tile must also carry for its convolutions to be correct. Pruning tiles that fall outside the template mask, dilated by each layer's receptive field, should cut the dominant early-layer cost roughly in proportion to the fraction of tiles removed. Packing one tile position across many subjects, extending the channel-by-channel batching idea, should further amortize the number of rotation keys.

One privacy point is essential. The mask must come from the template and never from the subject's own scan. A subject-specific mask would leak brain shape through the computation plan.

The contribution is not "read the dimensions from a JSON sidecar." That part is trivial. The contribution is a 3D packing and convolution schedule whose sparsity pattern is legitimately public, together with a measured accuracy–cost frontier for encrypted brain-age prediction. As far as the research file shows, this is the first such frontier.

Relation to Prior Work

Two lines of packing work supply the method we extend. HyPHEN arranges convolutions and slot layouts so that consecutive layers line up. It also fills slot gaps left by strided convolutions and splits high-resolution images to shrink the memory taken by weight plaintexts. Channel-By-Channel Packing packs one channel across 512 images, removes clean-up rotations, and cuts rotation keys from 105.6 GB to 6.91 GB. Both are 2D and dense. HyPHEN's gaps come from the arithmetic of strides, whereas ours come from anatomy, and anatomy cannot be exploited in natural images because each photograph's content differs. We borrow CBC's batching and HyPHEN's layer-to-layer layout alignment, and add tile pruning driven by the public template mask, in three dimensions.

Two neuroimaging papers motivate the gap without filling it. Secure Neuroimaging Analysis using Federated Learning with Homomorphic Encryption encrypts only the model weight vectors during federated averaging, while the scans stay in plaintext at each site. Privacy Preserving Image Registration runs affine registration on encrypted ADNI volumes and gets a 100x speedup from matrix partitioning, but it evaluates no non-linear network.

Two papers constrain the design. Benchmarking CNN on 3D Anatomical Brain MRI finds that heavy non-linear morphometry preprocessing helps in small data, and that is far too expensive to run under encryption. We therefore assume the client does only cheap affine alignment to the template, the "quasi-raw" setting, which the same paper shows becomes competitive near 10,000 training scans. Our training uses UK Biobank at that scale. Peng et al.'s lightweight SFCN is the natural model to encrypt: it has few layers, no fully connected layers, and aggressive pooling.

For the non-linear steps we follow the polynomial-activation tradition analysed in SecureNets. We also draw on activation-free polynomial backbones from Activation-Free Backbones for Image Recognition.

Research Program

The client affinely registers its T1 scan to a standard template at a chosen resolution. It then writes the result as a standardized derivative whose space and resolution entities, and the accompanying template mask, define a public "packing plan." This plan is a small artifact any BIDS-App container can regenerate deterministically. Everyone holding the plan derives the same tile grid, active-tile sets per layer, halo widths, rotation-key set, and modulus-chain depth. The plan is built once per template-and-network pair, not per patient.

The network is an SFCN variant retrained in plaintext on UK Biobank with low-degree polynomial activations. Average pooling replaces max pooling, because averaging is linear while max is not. Inputs are zeroed outside the template mask during training, so the encrypted computation matches the trained model exactly.

The encrypted engine uses RNS-CKKS, the encryption scheme for approximate real-number arithmetic. Within a tile, a 3×3×3 convolution becomes slot rotations by fixed offsets. Halos are carried explicitly, and the system re-tiles after each pooling stage, where the volume shrinks and the occupancy pattern changes. We will instrument every run for key-switching operations, bootstraps, peak memory, key bytes, and wall-clock time. We will also fit a simple cost model, with cost proportional to the sum over layers of active tiles times rotations per tile, and check whether that sum predicts measured latency across resolutions and masks.

Baselines

The naive reference is dense 3D packing. The full template box is flattened into ciphertexts with no tile pruning and no cross-subject batching, using the same network, encryption parameters, CPU server and thread count as our method. It isolates whether the proposal does anything at all.

The strongest existing method is a direct 3D extension of CBC packing: dense volumes, one tile position across a batch of subjects. CBC holds the best reported throughput and key-size result. On CIFAR-10 it reports ResNet-20 accuracy of 91.96% encrypted against 91.98% plaintext, a 15.28x rotation-key reduction, and 15.76 s amortized per image in batches of 512 (CBC Packing). We give it the identical model, batch size and hardware, and tune its encryption parameters with the same search budget as ours. Beating it means at least a 2x reduction in amortized latency and key size at no worse encrypted mean absolute error (MAE).

A second competitor is HyPHEN-style per-slice 2D inference: a 2D network over axial slices, packed with HyPHEN's layouts and run on the same hardware. It represents "reuse the mature 2D machinery." The file reports 14.69 s for ResNet-18 on ImageNet with HyPHEN, but no reported result found on Lacuna for encrypted brain-age MAE with any method.

The accuracy ceiling is plaintext SFCN at 2.14±0.05 years MAE on the UK Biobank test set (Peng et al.). We will report our own plaintext polynomial-activation model as the matched reference, so that losses from polynomial activations are separated from losses introduced by encryption.

The ablation that isolates the key ingredient keeps everything identical and toggles mask-driven tile pruning on and off. It is run at each resolution, with batching held fixed. A second ablation replaces the template mask with a box-crop. This tells us whether anatomy-shaped pruning beats simply cutting the corners.

Why not just…?

Why not just downsample the volume until dense packing is cheap? This is the real threat. At 4 mm, a 160×192×160 grid becomes 40×48×40, which fits in a handful of ciphertexts. Anatomical pruning may then buy little, and the accuracy lost to coarse resolution may be small. We do not argue around this. The resolution sweep reports the full curve of MAE against amortized latency for dense and pruned layouts at 1, 2, 3 and 4 mm. If dense packing at some coarse resolution dominates every pruned point on that curve, the core claim fails. If so, we will report that downsampling is the right engineering answer for encrypted brain age.

Why not just run ridge regression on the encrypted voxels? A linear model needs one multiplication and no bootstrapping, so it is nearly free. However, Dufumier et al. find that linear models are competitive only on morphometry-preprocessed inputs, which cannot be produced under encryption, and not on quasi-raw scans. We will run encrypted ridge on the same affinely registered, masked inputs and report its MAE. A deep encrypted model is justified only if it is clearly more accurate, with a gap well beyond the ±0.05-year seed spread reported for SFCN.

Why not just use secure multi-party computation, or keep the model on the hospital's own machine? Multi-party computation is fast per iteration, at 0.13 s for 2D affine registration in PPIR, but it needs continuous interactive exchange of 3D-volume-sized messages between parties. Local deployment requires handing the model weights to every client. Both are legitimate for some deployments. We address them by measuring total bytes communicated per subject for our protocol against a published SPDZ-style cost scaling for the same network. We do not claim encryption is universally better, only that it is the non-interactive option and that we make it affordable.

Proposed Experiments

Training and the main encrypted evaluation use the UK Biobank split of Peng et al. through their released code (UKBiobank_deep_pretrain). The encrypted test runs on a subsample large enough to estimate MAE to about ±0.1 years. We will also check that encrypted outputs match plaintext polynomial-model outputs on the full test set via simulation.

Transfer to new sites is tested on BHB-10K, with its 13 sites, and on PAC 2019. Both use the same template, so the packing plan is reused unchanged. This directly tests the premise that standardization makes the computation plan portable across sites.

The metrics are encrypted MAE, the gap between encrypted and plaintext predictions, per-subject and amortized latency, rotation-key bytes, peak server memory, and key-switching and bootstrapping counts.

The decisive checks are three. First, the pruning ablation must yield savings tracking the active-tile fraction predicted by the cost model. Second, the pruned layout must sit on the Pareto front of the resolution sweep against dense packing and dense 3D CBC. Third, encrypted MAE must stay within a few tenths of a year of the plaintext polynomial model while beating encrypted ridge. Failing the first check means the mechanism is wrong. Failing only the second means the mechanism is real but not worth its complexity, and we will say so.