Engineers predict whether a vertebra will fracture by building a computer model of it from a CT scan. The usual tool is finite element analysis, a simulation that splits the bone into small pieces, assigns each piece a stiffness from the scan brightness, and computes where stress builds up under load. Every such model starts from a segmentation: a person or a network outlines the bone on the scan. Two experts rarely draw the same outline. The disagreement is largest on thin, blurry structures, such as the 1–2 mm cortical shell and the vertebral endplates. Those are exactly the structures that carry load and where stress peaks.
This proposal measures how much expert-to-expert outline disagreement changes predicted vertebral strength and peak stress. It tests the hypothesis that the effect is largest at the thoracolumbar junction (T12–L1), the stiff-to-mobile transition where osteoporotic fractures cluster. It also tests which uncertainty-aware segmentation methods reproduce the mechanical spread humans actually produce, as opposed to the pixel-overlap spread they are currently judged on. Stated for a colleague in two sentences: run the same strength simulation on each expert's outline of the same vertebra and see how far the answers scatter. Then ask whether the cheap ways of faking that scatter get the mechanics right.
Core Claim
Our hypothesis has three parts. First, inter-observer variability yields a spread in predicted failure load that is meaningfully larger at T12 and L1 than at mid-lumbar levels. Second, the size of that spread is poorly predicted by the overlap scores the segmentation field optimizes. Third, how disagreement is arranged in space decides whether a synthetic uncertainty method gets the mechanics right. Coherent, rater-like shifts of a whole boundary behave differently in a simulation than independent per-voxel noise with the same local disagreement rate.
The study yields a reusable artifact and a falsifiable answer. The artifact is a multi-rater spine CT set paired with per-rater simulation outputs. The answer could easily be "the spread is negligible." That would itself be useful, because it would license the field's current practice of building models from one deterministic outline.
The motivating evidence comes from both sides. The reduced cortical bone density in vertebral bodies: risk for osteoporotic fractures? Insights from CT analysis segments cortical shell separately from inner trabecular bone in 144 patients. It finds the lowest cortical density at T12 (382.5 vs 444.6 mg/cm³ at L4), the highest fracture incidence at T12 (27%), and a shell that carries 45–75% of axial load. However, it never quantifies how much the shell boundary depends on who drew it. MDCT-Based Finite Element Analysis for the Prediction of Functional Spine Unit Strength—An In Vitro Study supplies the simulation method we reuse: scan-brightness-to-stiffness mapping, validated against cadaver compression tests. It too uses a single deterministic segmentation.
On the segmentation side, QUBIQ: Uncertainty Quantification for Biomedical Image Segmentation Challenge and the multi-rater models discussed below show that experts disagree systematically along ambiguous margins. They score that disagreement only geometrically. Nobody in the file has connected the two. We found no reported result on Lacuna for inter-observer spread in finite-element-predicted vertebral strength or peak stress.
The closest conceptual neighbour is Calibrate the inter-observer segmentation uncertainty via diagnosis-first principle. That work weights rater annotations by how well they support a 2D downstream diagnosis and reports AUC gains of up to 2.73% on REFUGE-2. We borrow its principle, that the downstream task, not overlap, should judge segmentation uncertainty, but we change both the downstream task and the goal. Our downstream quantity is a physical simulation output on 3D bone. We aim to measure and reproduce the human distribution of that output, not to pick a single best ground truth.
Research Program
The first step is data. We will draw about 40 multi-detector CT scans with adequate in-plane resolution from VerSe: A Vertebrae Labelling and Segmentation Benchmark. and CTSpine1K: A Large-Scale Dataset for Spinal Vertebrae Segmentation in Computed Tomography. We will oversample older patients and include a handful with transitional anatomy. Both benchmarks deliberately collapsed their annotators' work into one consensus mask, so we re-annotate. Five raters (two musculoskeletal radiologists, two spine surgeons, one trained biomechanics engineer) will independently segment T11 through L4. Each rater produces a whole-vertebra mask and a separate cortical-shell mask. Ten scans are redone by each rater after a washout period to measure within-rater inconsistency. Starting from a network pre-segmentation is tempting for speed, but it would anchor raters to the network's boundary. We therefore require from-scratch delineation of the shell and endplates on the levels of interest.
The second step is simulation. Each mask becomes a patient-specific vertebral body model. Stiffness is mapped from scan brightness following the MDCT paper. Because these scans lack calibration phantoms, we use phantomless density calibration from internal reference tissue. Calibration is fixed per scan, so rater comparisons see identical brightness-to-stiffness maps. We run two load cases. One is axial compression, the only case validated in the MDCT paper. The other is anterior flexion, which loads the anterior wall and endplate where thoracolumbar wedge fractures begin.
The outputs are predicted failure load and peak stress at the endplate and the anterior cortex. On a subset of 10 scans we build three-vertebra, two-disc units with wire-element ligaments, as in the MDCT paper, to check that conclusions survive the added soft tissue.
The central quantity is the per-vertebra mechanical spread. For vertebra $v$, let $F_{v,r}$ be the predicted failure load from rater $r$'s outline. We define the relative spread $s_v = \mathrm{sd}_r(F_{v,r}) / \mathrm{mean}_r(F_{v,r})$. The junction hypothesis is that the ratio of mean $s_v$ at T12/L1 to mean $s_v$ at L3/L4 exceeds one.
The third step compares methods. Each candidate uncertainty method generates $K$ sampled masks per vertebra, and each sample goes through the same simulation. We score a method by how closely its distribution of predicted failure loads matches the human one, using the energy distance:
$$ D_v = 2\,\mathbb{E}|F^{m}-F^{h}| - \mathbb{E}|F^{m}-F^{m\prime}| - \mathbb{E}|F^{h}-F^{h\prime}| $$
Here $F^{m}$ and $F^{m\prime}$ are independent failure loads from the method's samples, and $F^{h}$ and $F^{h\prime}$ are independent failure loads from human raters for the same vertebra. This is the one-dimensional, physics-space analogue of the generalized energy distance the segmentation field already reports on image masks. That makes the comparison between the mechanical and geometric versions direct.
Baselines
The naive reference is a single deterministic network outline fed to the simulation, with no uncertainty at all. We use nnU-Net trained on the CTSpine1K consensus labels; the top VerSe 2020 entry reached 91.72% Dice and 96.6% identification rate according to the VerSe paper. This baseline yields a point estimate and zero spread. Its role is to show how far that point estimate falls from the human-mean prediction. The comparison is fair because it uses the same scans and the same simulation.
The consensus baseline fuses the five rater masks with STAPLE and with majority vote, then runs one simulation. Simulation output is a nonlinear function of geometry, so the strength of the consensus outline need not equal the average strength across raters. We count our concern as real if this gap is larger than the within-rater repeat spread.
The strongest existing uncertainty method, judging by the file, is a deep ensemble of deterministic networks. QUBIQ found that ensembles frequently outperformed more elaborate probabilistic architectures at matching human uncertainty, with top teams averaging Dice between 0.80 and 0.85. We train five nnU-Nets with different seeds and data orderings on the same multi-rater training folds, one model per rater label set. Each gets the same epochs and augmentation as every other learned method.
The structured rater-model competitor is Pionono from Probabilistic Modeling of Inter- and Intra-observer Variability in Medical Image Segmentation. It gives each rater a learned Gaussian in an 8-dimensional latent space, whose mean captures the rater's systematic bias and whose spread captures inconsistency. It reported κ = 0.758 on Gleason 2019 against a prior best of 0.750. We also run the variational model of Variational Inference for Quantifying Inter-observer Variability in Segmentation of Anatomical Structures (continuous Dice 0.5372 on QUBIQ brain growth, versus 0.5083 for Bayes dropout), and Monte Carlo dropout. All learned methods are extended from 2D to 3D patches with matched parameter counts and an equal hyperparameter search budget. All draw the same number of samples per vertebra.
The engineering baseline involves no humans and no learning. It perturbs a single outline with random smooth boundary displacements whose amplitude is fitted to match the measured human disagreement rate. It also sweeps the bone-density threshold that separates cortex from trabecular bone.
The key ablation isolates spatial coherence. We take the human per-voxel disagreement map and draw independent per-voxel samples with exactly that local probability. Local disagreement is then identical to the humans', but each rater's boundary no longer shifts as a coherent whole. If this ablation matches human mechanical spread as well as coherent samples do, our claim that spatial structure matters is wrong.
A method counts as beating the ensemble if it achieves lower mean $D_v$ at T12/L1, with a paired bootstrap interval over vertebrae excluding zero. For the cadaver anchor, the bar is the MDCT paper's reported model-versus-experiment agreement: ρ = 0.79 and 576 N mean difference with ligaments.
Why not just…?
Why not just perturb one outline randomly and call that the uncertainty? This is the most serious threat, because it is cheap and standard in engineering. It is included above as a baseline with amplitude fitted to the human data. Our evidence that it may fall short is indirect. Uniform or random offsets ignore that the shell is only 1–2 mm thick (cortical density study). Stress also bottlenecks at discrete junctions rather than spreading evenly (pelvic floor finite element analysis). If the fitted perturbation matches the human failure-load distribution as well as the learned rater models do, we will report that as the main practical finding. In that case the recommendation becomes "calibrate a perturbation, skip the generative model."
Why not just use a deep ensemble, which QUBIQ already found competitive? It may well be enough. The structured rater models of the Latent Doctor Model, Pionono, and OMBA-UNet argue that ensembles capture model uncertainty more than systematic rater style. No one has checked which matters for mechanics. The energy-distance comparison settles this directly, and an ensemble win is a reportable outcome, not a failure.
Why not just simulate the consensus outline? Consensus is what VerSe and CTSpine1K provide, and it gives one number with no error bar. The Jensen-gap experiment tests whether that number is even the right centre of the distribution. The MDCT paper's 404 N (~16%) shift from adding ligaments shows that modest geometric changes at interfaces move predicted strength substantially.
Why not just automate the cortex with a density threshold and remove humans entirely? That replaces observer choice with threshold choice and does not remove it. The threshold sweep measures how much spread that choice produces. If the sweep's spread is far smaller than the human spread, a fixed automated threshold becomes a defensible standardization. Whether it is also correct cannot be judged without experimental failure loads, which the cadaver anchor below addresses.
Proposed Experiments and Decisive Checks
The first and gating check is whether the human spread matters at all. If the mean relative spread $s_v$ at T12/L1 is below roughly 5% of predicted strength and no larger than the within-rater repeat spread, the agenda stops there. We would publish a null result supporting single-outline modelling.
If the spread is substantial, the second check tests the junction hypothesis. We fit a mixed model of $s_v$ on vertebral level with a random effect per patient, adjusting for cortical density. The aim is to separate "junction anatomy" from "thin bone is hard to draw everywhere."
The third check asks whether geometry predicts mechanics. We correlate inter-rater Dice and surface distance with $s_v$ across vertebrae. A weak correlation would show that overlap-based uncertainty benchmarks rank methods poorly for biomechanical use.
The fourth check is the method comparison and the coherence ablation described above. We report $D_v$ per level and per load case, with flexion expected to separate methods more than compression.
Finally, if we can obtain scans and experimental failure loads for a cadaver series comparable to the MDCT cohort, through collaboration rather than the public benchmarks, we compare the human mechanical spread to the 576 N model-versus-experiment error. If observer spread is a large fraction of that error, segmentation subjectivity is a leading and so far unmeasured contributor to the accuracy ceiling of patient-specific spine strength prediction.
References
- The reduced cortical bone density in vertebral bodies: risk for osteoporotic fractures? Insights from CT analysis
- MDCT-Based Finite Element Analysis for the Prediction of Functional Spine Unit Strength—An In Vitro Study
- QUBIQ: Uncertainty Quantification for Biomedical Image Segmentation Challenge
- Calibrate the Inter-Observer Segmentation Uncertainty via Diagnosis-First Principle
- VerSe: A Vertebrae labelling and segmentation benchmark for multi-detector CT images
- CTSpine1K: A Large-Scale Dataset for Spinal Vertebrae Segmentation in Computed Tomography
- Probabilistic Modeling of Inter- and Intra-observer Variability in Medical Image Segmentation
- Variational inference for quantifying inter-observer variability in segmentation of anatomical structures
- Relationship between high intra-abdominal pressure and compliance of the pelvic floor support system in women without pelvic organ prolapse: A finite element analysis
- The Latent Doctor Model for Modeling Inter-Observer Variability
- Inter-Rater Uncertainty Quantification in Medical Image Segmentation via Rater-Specific Bayesian Neural Networks