Relative-Pose Fidelity for Synthetic Brain MRI

Generated brain MRIs can look real and still contain anatomy that is wrong. The most careful current check, the Wasserstein-Based Anatomical Brain Index (WASABI), segments each synthetic brain into 52 labeled structures and asks whether the distribution of their volumes matches real brains. Volume is only half of anatomy, though. A synthetic hippocampus can have exactly the right size and still sit at the wrong angle to the amygdala beside it, be twisted about its own long axis, or appear as a mirror image of itself. A volume-only metric cannot see any of these errors. We propose a second fidelity score for exactly this failure. It describes where each segmented structure sits and how it is oriented relative to its anatomical neighbours, and then compares the distribution of these relative positions between real and synthetic cohorts.

The way the score is built matters. We place a small coordinate frame on each structure, giving its position and orientation in 3D. The pose of one structure "as seen from" another is then an element of the group of 3D rigid motions, written $\mathrm{SE}(3)$. In this group, order matters: rotating and then translating is not the same as translating and then rotating. Group theorists call such groups non-Abelian. Writing relative pose as a group element gives a description that does not change when the whole head is moved or rotated in the scanner, so the score needs no registration to a template. It also keeps information that simpler descriptions discard, namely handedness and twist. The research question is whether this pose information exposes anatomical errors in current generators that volume metrics, image-texture metrics, and downstream-utility tests all miss.

Core Claim

Our claim is that relative pose between segmented brain structures is a separate axis of clinical fidelity, and that the right way to measure it is an optimal-transport distance over distributions of relative $\mathrm{SE}(3)$ poses. Concretely, segmentation gives each structure $k$ a frame $g_k = (R_k, t_k)$, where $R_k$ is a 3D rotation and $t_k$ is the structure's centroid. For each pair of neighbouring structures $(i,j)$ we form

$$h_{ij} = g_i^{-1} g_j ,$$

which is the pose of structure $j$ expressed in the coordinate frame of structure $i$. If the whole scan is moved by any rigid motion $g$, then $(g g_i)^{-1}(g g_j) = h_{ij}$, so the descriptor is exactly unchanged by head position. A cheaper "Abelian" substitute that works in scanner coordinates, using $t_j - t_i$ and $R_j R_i^{\top}$, changes whenever the head rotates. Our score, Pose-WASABI, is the 2-Wasserstein distance between the real and synthetic distributions of the collection $\{h_{ij}\}$, where the cost of moving one configuration onto another is

$$c(h, h') = \sum_{(i,j)} \Big( \theta\big(R_{ij}^{\top} R'_{ij}\big)^2 + \lambda \,\big\| t_{ij} - t'_{ij} \big\|^2 / s^2 \Big).$$

Here $\theta(\cdot)$ is the rotation angle of a rotation matrix (the geodesic distance on the rotation group), $t_{ij}$ is the translational part of $h_{ij}$, $s$ is a per-subject scale taken from intracranial volume (WASABI also normalizes by intracranial volume), and $\lambda$ trades degrees against normalized distance. The specific new prediction is that generators with similar volume scores will separate on this pose score, and that the separation will follow architecture. Models that build 3D volumes slice by slice, or that are conditioned in 2D, should show the largest pose distortions.

Research Program

The program has three parts.

Part 1: build the descriptor. We take the parcellation used by Anatomical Similarity as a New Metric to Evaluate Brain Generative Models, which introduces WASABI. That paper segments 52 bilateral regions with SynthSeg, a parcellation network that works across MRI contrasts. We keep its segmenter and its normalization so that our score differs from WASABI only in what it measures. Each structure's frame comes from its centroid and principal axes.

The obvious weakness is that principal axes have arbitrary signs and become unstable for near-spherical structures. We fix the signs by anatomical anchoring: each axis is pointed toward a fixed neighbouring structure. We exclude a structure from orientation terms when the eigenvalue gap of its shape tensor is too small to define an axis reliably, and we report bootstrap frame stability for every structure we keep. Pairs are restricted to a fixed anatomical adjacency graph, for example hippocampus–amygdala, thalamus–lateral ventricle, caudate–putamen, and left–right homologues. This keeps the score interpretable and its dimension modest.

Part 2: estimate the distance. The distributions live on a curved space, so WASABI's closed-form Gaussian Wasserstein formula does not apply directly. We compute entropy-regularized optimal transport between empirical samples using the cost above. As a cheaper approximation, we also map each $h_{ij}$ to log-coordinates of the group and fit Gaussians there. We report whether the two estimates agree.

Part 3: validate. We test the score against controlled ground truth, test its invariance and biological sensitivity, and then apply it to published generators.

The three research directions play unequal roles. Region-of-interest segmentation is the mechanism that turns a volume into a set of rigid bodies. Non-Abelian group structure supplies the descriptor and its invariance guarantee. Clinical fidelity is the setting where the score has to earn its place.

Prior work shapes the design in three ways. Regular SE(3) Group Convolutions for Volumetric Medical Image Analysis builds rotation-equivariant 3D networks. It motivates this work by showing that non-equivariant 3D models lose more than 62% accuracy on rotated OrganMNIST3D. It also constrains it: the same paper finds that strict global equivariance can hurt when scans have a standard clinical orientation. Our score avoids that trade-off. It is invariant to global head pose, yet it keeps every relative orientation between structures, which is where the anatomical information lies. Brain Tumor Type Classification via Capsule Networks supports a related point: once a structure is isolated by segmentation, modelling its pose explicitly helps, with 86.56% accuracy for capsule networks on segmented tumours versus 72.13% on whole brains. We reuse that intuition for measurement rather than classification.

Baselines

Do-nothing reference. The naive reference point is to report no pose measure at all, only volume-based WASABI. On the UK Biobank reference cohort, WASABI ranks MedSyn at 3.29, BrainSynth at 5.03 and HA-GAN at 22.63, as reported in Anatomical Similarity as a New Metric to Evaluate Brain Generative Models. WASABI is the strongest existing anatomical fidelity metric in this space, and Anatomically Guided Latent Diffusion for Brain MRI Progression Modeling already uses it to show gains from segmentation-guided training. We run WASABI with the same SynthSeg outputs, the same reference cohort and the same sample sizes as our score. Pose-WASABI beats it if it detects perturbations that leave WASABI unchanged, and if it produces generator rankings that disagree with WASABI in ways confirmed by independent checks (described under Proposed Experiments). Pose-WASABI would add nothing if it merely tracks WASABI.

Registration-based check. The Realistic Atlas Score (RAS) from Modelling the Distribution of 3D Brain MRI using a 2D Slice VAE segments a volume, registers it non-linearly to an atlas and computes label overlap. Its authors note that blurry, average-looking volumes can inflate it, and we found no reported RAS number on Lacuna. We implement RAS on identical inputs, with both rigid-only and non-linear registration.

Image-statistics reference. Fréchet Inception Distance (FID), a feature-distribution distance based on a natural-image classifier, is the image-level reference.

Strongest simple competitor. The most dangerous baseline is a hand-built invariant feature vector added to WASABI. For each adjacent pair it contains the centroid distance and the unsigned angles between principal axes. These features are also unchanged by head motion and are trivial to compute, so they are the strongest test of whether the group machinery is needed. They are computed from exactly the same frames, given the same Wasserstein estimator, and given the same tuning budget.

Ablation of the key ingredient. The ablation that isolates the non-Abelian ingredient is the scanner-frame variant described in the Core Claim, which uses $t_j - t_i$ and $R_j R_i^{\top}$ in world coordinates. It keeps all the information in our descriptor but loses the invariance.

Why not just…?

Why not just add centroid distances and axis angles to WASABI? This is a real threat, and we treat it as one. Pairwise distances and unsigned angles are unchanged by rotation, but they are not a complete description of a configuration. They cannot tell a structure from its mirror image, and they miss twist about a shared axis, because both kinds of change preserve every pairwise distance and every unsigned angle. The relative pose $h_{ij}$ keeps both.

Whether these errors actually occur in generated brains is an empirical question. The perturbation experiment below injects each error type separately. If the feature-vector baseline matches Pose-WASABI on detection and on generator ranking across real generators, we will report that the cheap descriptor is sufficient. The contribution then becomes the evidence that relative pose matters at all, not the group formulation.

Why not just rigidly register everything to a template and compare centroids? After rigid registration, comparing centroids is our score computed in one particular global frame. That frame comes from a registration step that can fail, and non-linear registration can absorb exactly the local misplacements we want to detect. Volokitin et al. note that registration-based scores reward smooth outputs. We test this directly by applying random global rotations and simulated registration failures to real and synthetic volumes. Pose-WASABI should stay stable by construction, while registered-centroid scores should drift.

Why not just use FID or downstream utility? WASABI's authors show on ADNI that FID is driven by scanner and site differences rather than anatomy and fails to rank Alzheimer's-versus-control above sex differences. Anatomically-Controllable Medical Image Generation with Segmentation-Guided Diffusion Models reports that synthetic data can train a segmenter with a Dice drop of at most 0.04, which is a coarse signal that a local twist of one structure is unlikely to move. We include both as references and expect them to be insensitive to injected pose errors.

Proposed Experiments

Controlled perturbation test. On real UK Biobank and ADNI scans, we apply smooth local deformations that rotate a single structure about its own centroid by 5–20 degrees, translate it, twist it, or mirror it, while preserving its volume. Volume preservation makes WASABI nearly blind to these changes by design. We measure detection power as a function of perturbation size for Pose-WASABI, the invariant-feature baseline, the scanner-frame ablation, RAS and FID.

Invariance test. We apply random rigid motions to whole volumes, sampled both within the clinically realistic range and over the full rotation group. A good score should change only by the small amount attributable to resampling and re-segmentation. The scanner-frame ablation should fail this test, and the comparison quantifies the cost of treating pose as Abelian.

Biological validity. Following WASABI's protocol on ADNI, we ask whether Pose-WASABI ranks control-versus-Alzheimer's differences above sex differences. We present this as a hypothesis, not a certainty: ventricular enlargement may shift neighbouring structures. A null result would itself be informative about how specific the score is to errors rather than to disease.

Generator audit. We score the generators WASABI evaluated (MedSyn, BrainSynth, HA-GAN), BrLP and AG-LDM where samples or code are available, and a slice-based VAE in the style of Volokitin et al. trained on the Human Connectome Project data. The decisive check is whether any pair of generators reverses order between WASABI and Pose-WASABI. If one does, a blinded neuroradiologist review of the highest-scoring outlier configurations will judge whether the pose score flags real anatomical errors. Slice-wise generators are predicted to rank worst.

Segmenter dependence. Like WASABI, our score inherits failures of the segmenter. We repeat the main results with an independently trained second segmenter and report how often rankings change.