Threshold-Aware Stopping for Second Reads in CT Body Composition

Automated CT tools now measure muscle and fat at the third lumbar vertebra (L3) with overlap scores above 0.95. Yet the clinical decision built on these measurements is a yes/no call against a sex-specific cutoff. Examples are "is this patient sarcopenic?" and "did this man lose at least 7.23% of his muscle index in a month?" Near the cutoff, the disagreement between two expert tracers can be larger than the margin that decides the label. Today's pipelines handle this in one of two crude ways. Some trust the single automated mask. Others hand every case to a human for correction, which brings back the fatigue and rater-to-rater disagreement that automation was meant to remove.

We propose treating the choice between "accept the label" and "ask for another human read" as a finite-horizon sequential test. Each read, whether automated or human, is a noisy sample of the true tissue area. The system keeps asking for reads only while the patient's label against their own sex-specific cutoff is still in doubt. The confidence it demands shrinks as the reading budget runs out, which is what a "collapsing boundary" means. In two sentences: easy cases far from the cutoff get zero human reads, and borderline cases get as many reads as it takes to settle which side of the cutoff they fall on, up to a fixed budget. Because the stopping rule is learned in the units of the clinical decision rather than in pixel overlap, it should match the label accuracy of universal double-reading at a fraction of the reader time.

Core Claim

Body composition errors that matter clinically are errors in the label, not in the mask. Their probability depends on two things: how far the patient's measurement lies from the cutoff for their sex, and how much raters disagree on this particular scan. The second quantity varies from case to case. Scans of obese patients, scans with streak artifacts, and scans with fascial planes that are hard to see all spread raters further apart. The core claim is that a stopping rule driven by distance to the sex-specific cutoff, scaled by per-case rater spread, reduces human reading effort for a given label error rate compared with three simpler rules. The first is a fixed margin band around the cutoff. The second is a classical sequential test with a constant threshold. The third is a uniform, sex-blind cutoff.

This claim stands or falls on two measurable facts. The first is whether rater spread is in fact heteroscedastic, that is, whether it varies meaningfully across cases. The second is whether sex-specific cutoffs move a meaningful number of patients into or out of the ambiguous zone. If rater spread turns out to be roughly constant across cases, the method reduces to a fixed band. We will report that outcome as the finding rather than defend a more complex rule.

Relation to Prior Work

Three groups of papers bear on this, each in a different role.

The body composition automation papers motivate the gap. One pipeline uses a DenseNet to pick the L3 slice and a U-Net to segment it, reaching Dice 0.95–0.98 and R = 0.99 against manual tracing (Fully-Automated Analysis of Body Composition from CT in Cancer Patients Using Convolutional Neural Networks). It treats the manual masks as a single ground truth, even though its two cohorts were traced with different software (Slice-O-Matic and Osirix). A second tool pairs multi-view localization with a 2D nnU-Net, a self-configuring U-Net variant. It reports Dice above 0.95 and a 95th-percentile Hausdorff distance near 2 mm, yet it still includes an interactive refinement interface for ambiguous boundaries (Ai-Driven Automated Tool for Abdominal CT Body Composition Analysis in Gastrointestinal Cancer Management). That tool leaves the question of when to refine entirely to the user, and that is exactly the question we formalize.

The sex-stratification papers supply the decision target and set its stakes. In a perioperative gastric cancer cohort, significant muscle loss was defined as at least 7.23% in men and at least 6.57% in women over 30 days. That loss carried a hazard ratio of 11.231 for death (Preoperative sarcopenia and postoperative accelerated muscle loss negatively impact survival after resection of locally advanced gastric cancer). In kidney CT, attenuation cutoffs were likewise found to diverge by sex (AI-based segmentation of renal enhanced CT images for quantitative evaluate of chronic kidney disease). Both papers apply the cutoffs downstream, to fixed numbers, with no feedback into how carefully those numbers were measured. Muscle loss is a ratio of two scans, so tracing error enters twice. This makes it a natural case where a 0.66-percentage-point difference between the sex-specific cutoffs is comparable to plausible measurement noise.

The multi-rater and stopping papers supply the method. Pionono gives each rater a Gaussian distribution in a latent space: the mean captures that rater's systematic bias and the variance captures their inconsistency over time (Probabilistic Modeling of Inter- and Intra-observer Variability in Medical Image Segmentation). We reuse this idea to generate a predictive distribution of tissue area for each scan. DiFF argues that the right consensus mask is the one that best serves diagnosis, not the geometric average (Calibrate the inter-observer segmentation uncertainty via diagnosis-first principle). We adopt the same diagnosis-first stance. The difference is that DiFF learns a single static weighting of experts in one pass. We instead decide how many expert reads to buy for each case. FIRMBOUND learns collapsing stopping boundaries for finite-horizon sequential tests by estimating the risk of continuing (Learning the Optimal Stopping for Early Classification within Finite Horizons via Sequential Probability Ratio Test). We borrow its way of learning that risk. FIRMBOUND was only tested on time series such as sensor signals and video, and it assumes the training and test distributions match; we have to check that assumption across hospitals. Theory showing that optimal stopping regions narrow over time under deadline pressure (Sequential Hypothesis Testing under Stochastic Deadlines) justifies the shape we expect the learned boundary to take. We do not take the radical step of segmenting slice by slice under a stopping rule, which the research file lists as an open gap. The costly resource in this setting is reader time, not GPU seconds, and the automated models already run in about one second.

Research Program

Multi-rater L3 subset. The research file contains no public body composition dataset with multiple independent tracings per scan, so the first deliverable is one. We will have four trained readers independently trace muscle, visceral fat, and subcutaneous fat on about 150 L3 slices. These slices will be drawn from the pancreatic and lymphoma cohorts used by Fully-Automated Analysis of Body Composition from CT in Cancer Patients Using Convolutional Neural Networks, enriched for patients near the sarcopenia cutoffs and balanced by sex. Where institutional access allows, we will add paired pre- and post-operative scans from cohorts like the gastric cancer trials, so that muscle loss can be measured. At roughly 20 minutes per tracing, this is about 200 reader-hours. The reference label for each case is the cutoff decision made on the mean of all four readers' areas. This gives a reference that is noisy but better than any single read.

Per-case predictive distribution. We train an nnU-Net with a Pionono-style latent variable for each rater. Sampling that latent variable produces a distribution of muscle area for each scan, and hence a distribution of skeletal muscle index (SMI) or muscle-loss percentage. The model is trained on all single-rater data plus the multi-rater subset. From this distribution we get the prior probability that the patient lies above their sex-specific cutoff.

Sequential test. Let $p_t$ be the posterior probability that the patient's true value exceeds the cutoff $c_s$ for their sex $s$, after $t$ human reads, with $t$ running from 0 to a budget $T$ of 3. Each human read updates $p_t$ through a measurement model whose noise is estimated per rater from the multi-rater subset. The rule stops and assigns a label as soon as

$$p_t \ge 1 - b_t \quad \text{or} \quad p_t \le b_t,$$

where $b_t$ is the allowed error at step $t$. We do not hand-pick $b_t$. Following FIRMBOUND, we learn it by fitting the expected risk of continuing, given a cost $\lambda$ per read and a cost per wrong label, and solving backward from the forced decision at $T$. Because the posterior is computed relative to $c_s$, the same image evidence can call for a stop in a man and a continuation in a woman. This is how sex-specific normalization enters the inference step itself rather than being applied afterward. Sweeping $\lambda$ traces out a curve of reads per case against label error.

Transfer check. We will learn $b_t$ on one institution's cases and test it on the other's. This directly probes the stationarity assumption that the FIRMBOUND paper flags as a known limitation.

Baselines

The naive reference is the deterministic nnU-Net with no human review: a single automated mask, compared against the sex-specific cutoff. The strongest published numbers for this kind of pipeline are mask-level, not label-level. Fully-Automated Analysis of Body Composition from CT in Cancer Patients Using Convolutional Neural Networks reports Dice 0.95–0.98 and R = 0.99, and Ai-Driven Automated Tool for Abdominal CT Body Composition Analysis in Gastrointestinal Cancer Management reports Dice above 0.95. We found no result on Lacuna for label-level sarcopenia or muscle-loss error rates of automated pipelines against a multi-rater reference. Producing that number is itself a contribution. We will train the nnU-Net on the same data as our model, with the same compute.

The expensive reference is universal review: every case gets one human read, or the full three, and the label is set from the mean. This bounds what review can achieve and fixes the reader cost the proposal has to undercut.

The strongest simple competitor is a margin band that uses per-case uncertainty. Any case whose predicted value lies within $k$ predictive standard deviations of $c_s$ gets reads one at a time until it leaves the band or the budget runs out, with $k$ tuned on the same validation split as $\lambda$. A cruder version uses a fixed absolute band, such as ±1 SMI unit, with no per-case uncertainty. A constant-threshold sequential test, the classical form from Wald, uses the same posterior with $b_t$ held fixed.

The key ablation isolates sex normalization. We run the full system with a single pooled cutoff and evaluate it against the sex-specific reference labels. A second ablation replaces Pionono-style rater modeling with Monte Carlo dropout to test whether modeling rater style matters.

The proposal beats these baselines if, at matched label error, it uses fewer human reads per case than the uncertainty-aware band. The bar is that its curve of reads against error lies strictly below the band's on held-out data, with bootstrap confidence intervals excluding zero. A second requirement is that, on the muscle-loss endpoint, its labels keep a survival hazard ratio no worse than labels from universal review.

Why not just…?

Why not just flag cases near the cutoff for a second read? This is the real threat, and it is why the uncertainty-aware band is the main baseline rather than a straw man. If rater spread is about the same for every scan, distance to the cutoff alone captures nearly all the information, and the band will match the sequential rule. Our first experiment measures how much rater spread varies across cases on the multi-rater subset. If the case-to-case variation is small, we will report that the simple band suffices and stop there. The sequential rule can only win in two situations: when some scans are much harder to trace than others, and when a second read sometimes fails to settle the label, so that deciding whether to request a third read matters.

Why not just use majority voting or STAPLE over several readers? These consensus methods answer "what is the mask given these reads?", not "how many reads do we need?". They also assume every case was read by several people, which is the cost we are trying to avoid. On histopathology, Pionono beat STAPLE-style consensus, with a Cohen's kappa of 0.758 on Gleason 2019 compared with 0.750 for the previous best method (Probabilistic Modeling of Inter- and Intra-observer Variability in Medical Image Segmentation). This suggests rater style carries usable signal. We test this directly through the dropout ablation, which removes rater-style modeling.

Why not just apply sex-specific lookup tables after a deterministic segmentation? Applying the cutoffs afterward does give the right threshold. What it cannot do is decide where reading effort should go. A segmentation error of a given size matters when it straddles the cutoff for that patient's sex and does not matter otherwise. The pooled-cutoff ablation measures how much review allocation and label accuracy change when the stopping rule ignores sex. If the change is negligible, sex-aware stopping adds nothing beyond the lookup table, and we will say so.

Why not just use a fixed confidence threshold instead of a collapsing one? With a budget of only three reads, the difference may be small. Theory predicts that the optimal boundary narrows near a deadline (Sequential Hypothesis Testing under Stochastic Deadlines), but in a three-step problem a fixed threshold could come close. The comparison with the constant-threshold test settles this. A small gap would push us to recommend the simpler rule.

Decisive Checks

The first check is case-to-case variation in rater spread on the four-reader subset, reported separately by sex and by visceral fat level. This determines whether anything beyond a margin band can help at all.

The second check is the main comparison: curves of human reads per case against label error for sarcopenia and for muscle loss, comparing the full system, the uncertainty-aware band, the fixed band, the constant-threshold test, no review, and universal review. The stopping boundary is learned on one institution and tested on the other.

The third check is clinical. On cohorts with survival follow-up, we compare hazard ratios for muscle loss when labels come from each policy, against the published 11.231 (Preoperative sarcopenia and postoperative accelerated muscle loss negatively impact survival after resection of locally advanced gastric cancer). A policy that saves reads but weakens this association has failed.

As a public sanity check of the stopping machinery on an existing multi-rater benchmark, we apply the same rule to optic cup-to-disc ratio on REFUGE-2, which is a threshold decision derived from a segmentation. We use the DiFF setting (Calibrate the inter-observer segmentation uncertainty via diagnosis-first principle) and its reported diagnostic AUC gain of up to 2.73% over majority voting as context. No sex stratification is applied in this check.