Most three-dimensional spine imaging is done with the patient lying down, but patients stand up in daily life. Under body weight the spine changes shape: discs compress, curves deepen, and vertebrae rotate. Recent networks try to close this gap by turning a pair of standing X-rays into a 3D spine. Their output is a voxel volume or a dense deformation field, so a surgeon cannot see what the network decided happened between lying and standing. We propose to make that decision the explicit output. The model reads the standing radiographs and predicts a short list of named clinical quantities: for each pair of adjacent vertebrae, the flexion, lateral tilt, axial rotation and translation that carry the patient's own supine CT into the standing pose. The 3D standing spine is then built by moving the patient's CT vertebrae as rigid bodies according to those numbers. Because the numbers are the output, a clinician can read them, see which ones the model is unsure about, and correct one, such as a lumbar lordosis measured by hand, and the reconstruction updates.
The opportunity is concrete. Supine imaging misstates alignment by clinically meaningful amounts that differ from patient to patient. In the lower limb, the mechanical axis differed by 2.1 ± 1.7° between supine CT and standing radiographs, ranging up to 6.2° (Three-dimensional preoperative planning in the weight-bearing state). Supine DXA underestimates scoliotic curvature (Bourigault et al.), and supine MRI misstates cervical sagittal balance (Zhong et al.). Existing posture reconstructions cannot say which of these changes they inferred.
Core Claim
The change from lying to standing in the spine is mostly motion at the joints, not reshaping of the bones. A vertebral body barely deforms under physiological load, while discs and facet joints do. The transformation therefore has a natural low-dimensional, clinically named parameterization: a chain of intersegmental rigid motions. Here "rigid" means the vertebra is moved and turned but not reshaped, and "intersegmental" means the motion is measured between neighbouring levels.
We write the pose of vertebra $i$ as $T_i = T_{i-1} J(\theta_i)$. Here $T_i$ is the vertebra's 3D rigid transform in the standing frame, and $J(\theta_i)$ is the joint transform built from $\theta_i$, a six-number vector of segmental flexion, lateral bend, axial rotation and three translations between levels $i-1$ and $i$. A network $g$ maps the two standing radiographs $x$ and the supine CT vertebrae $V$ to $\hat\theta = g(x, V)$ with a per-parameter uncertainty. The reconstruction is the CT vertebrae placed at poses $T(\hat\theta)$.
Training uses simulated data, described below, where the true $\theta$ is known. It also enforces a projection consistency term:
$$\mathcal{L} = \lambda \,\|\hat\theta - \theta\|^2 + \sum_{v\in\{\text{AP},\text{LAT}\}} d\big(P_v(V, T(\hat\theta)),\, x_v\big),$$
In this loss, $P_v$ renders a digitally reconstructed radiograph (a simulated X-ray computed from CT) in view $v$, $d$ is a local image-similarity loss, and $\lambda$ weights the direct parameter supervision.
This is a concept bottleneck in the sense of Concept Bottleneck Models, which route predictions through human-readable intermediate concepts that clinicians can edit at test time. We differ from that work in two ways. Our concepts are continuous geometric quantities rather than annotated categories. They also need no expert labelling, because a simulator produces them for free. That removes the annotation cost that limits concept bottlenecks in volumetric imaging.
The central empirical claim is that this bottleneck costs little or no shape accuracy against black-box deformation networks. In return it provides three things those networks lack: posture parameters that can be audited, uncertainty that flags failures on real radiographs, and test-time intervention.
Relation to Prior Work
Three reconstruction networks motivate the gap. TransVert expands biplanar views into 3D with anisotropic convolutions. ReVerteR inflates 2D features into per-vertebra volumes. TransVert+ deforms vertebral templates with an affine decoder followed by a voxel-wise deformable decoder. All three output voxels, and all three are trained on radiographs simulated from supine CTs. That means their "standing" training targets are actually supine postures. We differ in both respects. Our output is an explicit joint chain, and our simulator deliberately perturbs the supine poses into sampled standing-like configurations, so the training data contains posture change.
TransVert+ is the closest competitor. Its affine stage is a crude single global transform, and its deformable stage is where interpretability is lost. We replace the deformable stage with a per-level chain.
Two works supply method. Wang et al. showed that regressing coefficients of a PCA shape model instead of raw landmarks removes anatomically impossible outputs on standing X-rays. We carry the same idea into 3D, using joint angles rather than PCA modes so each coefficient has a clinical name. The weight-bearing planning study (Three-dimensional preoperative planning) supplies the classical alternative, intensity-based 2D-3D registration. It is accurate on a phantom but needs manual landmarks and 10–15 minutes per patient.
Two works constrain the design. Interpretable Vertebral Fracture Diagnosis found that post-hoc dissection of a 3D network recovers radiological concepts, with 7 of the top 10 units mapped to signs. However, those concepts were learned only from supine CT and cannot be edited. Seg4Reg+ uses heatmap-based attention constraints, which show where the model looks but not what geometry it infers. We take both as evidence that editable geometric quantities are what is missing. The review by Rueckert & Schnabel frames the general case for hybrid models that combine learned components with explicit physical models, especially under domain shift.
Research Program
Simulator. From each VerSe CT (VerSe benchmark) we take the segmented vertebrae as rigid bodies. We sample intersegmental parameter vectors from ranges set by published standing-versus-supine differences and by scoliotic axial rotation. We then render biplanar radiographs with realistic noise, rib and pelvis overlap, and contrast degradation. This yields exact ground-truth $\theta$, exact 3D standing geometry, and paired supine input. Disc soft tissue is approximated by interpolating the intensity between moved vertebrae. Because this approximation is imperfect, rendered images are compared only on bone-weighted regions.
Predictor. An image encoder reads both radiographs. A sequence model runs along the spine and predicts $\theta_i$ with Gaussian uncertainty for each level, which lets neighbouring levels share information. This addresses the neighbour-confusion failure seen in 2D landmark methods. An optional refinement step then takes a few gradient steps on the projection term alone. This gives the classical registration objective an automatic starting point.
Intervention. At test time a clinician may fix any subset of $\theta$, for example an L1–S1 lordosis read off the lateral film. The remaining parameters are re-estimated conditionally, both through the sequence model and through refinement.
Side-channel ablation. We train a variant that adds a small residual voxel deformation on top of the chain. If that residual improves accuracy substantially, the rigid-chain concepts are incomplete, and we report this openly. Fractured or wedged vertebrae that deform under load are the expected place for this to happen.
Baselines
The naive reference is doing nothing: the supine CT alone, aligned to the radiographs by one global rigid transform. This measures how much posture change exists to recover. A second naive reference adds a fixed population-mean segmental offset. The strongest learned competitor is TransVert+. On simulated radiographs it reports Dice 0.9510, Hausdorff distance 5.70 mm and normalized Wasserstein distance 0.1797 (TransVert+). We also run ReVerteR, which reports Dice 0.7938 on VerSe-derived simulated radiographs (ReVerteR).
We retrain both on our pose-perturbed simulator with the same CT splits, the same augmentations and a matched GPU-hour tuning budget, so neither side benefits from a different training distribution. We also report them under their original supine-only simulation to isolate the effect of the training data. Joint angles for these baselines are extracted from their output volumes by fitting each CT vertebra rigidly to its predicted voxels.
The third baseline is classical intensity-based 2D-3D registration in the style of the weight-bearing planning study, using local normalized cross-correlation and bounded derivative-free optimization. It is run per vertebra with manual-quality initialization from ground-truth centroids. This is its best case. Its reported accuracy is a maximum of 1.1 mm and 1.2° on a phantom, at 10–15 minutes per case (Three-dimensional preoperative planning). No reported result was found on Lacuna for this registration method applied to the spine.
Beating the learned baselines means intersegmental angle error and Hausdorff distance no worse than TransVert+ retrained, together with uncertainty that is meaningfully better calibrated. Beating classical registration means matching its accuracy without manual initialization, in seconds, with a lower rate of optimization failures.
Why not just…?
Why not just use classical 2D-3D registration, which is already transparent? This is the real threat. Classical registration optimizes interpretable rigid poses already, so our interpretability advantage over it is small. What it lacks is automatic, reliable initialization. The weight-bearing study depended on manual landmark picking, and overlapping structures push the optimizer outside the range from which it can converge. Our position is that the learned chain is an amortized, automatic initializer for the same objective, with uncertainty attached. The experiment tests exactly this. We compare classical registration from automatic initialization, classical registration from ideal initialization, and our predictor with and without refinement, reporting failure rate and time. If automatically initialized classical registration fails rarely, the learned component is not justified, and we will say so.
Why not just run TransVert+ and measure angles from its output? Angles can indeed be measured afterward, and we do exactly that as a baseline. However, the output volume may contain vertebral shapes the network invented rather than the patient's own bones. An edit to one angle also has no route back into the prediction. The comparison of angle accuracy and calibration between post-hoc-extracted and bottleneck angles settles whether the bottleneck costs accuracy. The intervention experiment settles whether it adds value.
Why not just measure Cobb angles on the standing radiograph with a 2D method? For the coronal Cobb angle alone, this is likely sufficient. Seg4Reg+ reports 7.32% SMAPE (symmetric mean absolute percentage error) on the AASCE challenge (Seg4Reg+). We do not expect to beat it and aim only for non-inferiority in Cobb angles projected from our 3D output. Our target is different: axial rotation, the segmental change from supine to standing, and a standing 3D model that can be used for planning. None of these are visible to a 2D method.
Why not just add a fixed correction offset? Patient-to-patient spread rules this out. In the lower limb the discrepancy ranges from 0° to 6.2°. The mean-offset baseline above quantifies how much of the spine's variance an offset leaves unexplained.
Proposed Experiments
On held-out VerSe patients, never seen during training, with sampled standing poses, we measure four things: intersegmental angle error per degree of freedom, per-vertebra registration error in millimetres, Dice and Hausdorff distance for comparison with prior reports, and the fraction of levels with errors above 3°.
The intervention experiment fixes $k$ parameters to their true values, either chosen at random or chosen as the ones the model is least certain about. We then measure how much error drops at the remaining levels. This mirrors the concept-bottleneck result on knee X-rays, where correcting two concepts reduced grading error from above 0.4 to about 0.3.
For the audit of simulated-to-real transfer, there is no standing 3D ground truth, so we use indirect checks. On real standing radiographs, we measure the reprojection error of clinician-annotated vertebral corners and the agreement of projected Cobb angles with the AASCE annotations. Because AASCE has no paired CT, a population-average vertebral shape stands in for the patient's CT there. We also test whether predicted uncertainty ranks the levels with the largest reprojection error.
A decisive result for the proposal requires three outcomes. First, angle and surface error are within noise of retrained TransVert+. Second, uncertainty-guided intervention on two parameters reduces the remaining error clearly more than random intervention. Third, the side-channel residual improves accuracy only on fractured vertebrae. If the residual helps everywhere, the rigid-chain assumption is wrong and the claim fails.
References
- Three-dimensional preoperative planning in the weight-bearing state: validation and clinical evaluation
- Identifying scoliosis in a population-based adult cohort: automation of a validated method based on total body dual energy X-ray absorptiometry scans
- Sagittal balance parameters measurement on cervical spine MR images based on superpixel segmentation
- Concept Bottleneck Models
- Inferring the 3D Standing Spine Posture from 2D Radiographs
- Automatic 3D reconstruction of vertebrae from orthogonal bi-planar radiographs
- Anatomy-Aware Inference of the 3D Standing Spine Posture from 2D Radiographs
- Accurate Scoliosis Vertebral Landmark Localization on X-ray Images via Shape-constrained Multi-stage Cascaded CNNs
- Interpretable Vertebral Fracture Diagnosis
- Seg4Reg+: Consistency Learning Between Spine Segmentation and Cobb Angle Regression
- Model-Based and Data-Driven Strategies in Medical Image Computing
- VerSe: A Vertebrae labelling and segmentation benchmark for multi-detector CT images