Medical image segmentation means drawing pixel-accurate outlines of lesions. Today it depends on specialists tracing boundaries by hand, and that labor is the main thing slowing clinical AI. The cheap substitute is to train on numbers hospitals already record: a tumor percentage on a pathology report, a worst-severity grade for a patient, a serum tumor marker. Every current method that does this quietly assumes the recorded number is a simple summary of what the image shows, either proportional to lesion area or equal to the worst region. Clinical biochemistry shows that assumption is often false. A small lesion's secretions are diluted in liters of blood. Lesions behind the blood-brain barrier may not show up in serum at all. Systemic inflammation can push a "local" fluid marker sky-high with no tumor present. When the number and the image disagree for these reasons, a segmentation network trained on the number learns the wrong outlines.
We propose to treat each routine clinical number as a measurement produced by a small, explicit transport model, and to train segmentation through that model. The model has three ingredients: dilution with saturation, a per-compartment barrier coefficient, and an additive confounding term driven by observed inflammation markers. The network predicts a per-pixel lesion probability map. A differentiable measurement layer turns that map into the blood or fluid value we should expect to observe. The loss compares expected with observed values, so disagreement between number and image is explained by physiology instead of being absorbed into the outlines. To keep the measurement model from absorbing everything, it is pinned down by cheap anchors that hospitals already have: healthy scans, which must produce zero local signal, and radiologists' routine caliper measurements of lesion diameter, known as RECIST lines.
Core Claim
The claim is narrow and falsifiable. When a weak label is generated through a non-identity, compartment-dependent and confounded channel, segmentation trained under the usual proportional or maximum assumptions degrades in a predictable way. Modelling the channel explicitly, with a few interpretable parameters and anchors that make it identifiable, recovers most of that loss. It does so without discarding the discordant patients, which are exactly the ones where the image matters most.
Where This Sits in Prior Work
The motivating evidence comes from clinical studies of divergence between blood and tissue. Oosting et al. imaged 125 metastatic renal cancer lesions with a radiolabeled antibody tracer. Plasma VEGF-A, the circulating marker, did not track lesion uptake. Lesion uptake itself varied enormously within single patients, and lesions with high uptake progressed far more slowly (hazard ratio 0.22). Trapé et al. handled the reverse problem in ascites, where systemic markers leak into local fluid. They required a fluid-to-serum ratio above 1.2 and restricted analysis to non-inflamed patients, which reached 81.4% sensitivity at 98.7% specificity. They also documented a benign pancreatitis effusion with CA19-9 of 6,370 ku/l. Their rule is in effect a hand-built observation model with a dilution term and an inflammation gate. We turn it into a learnable layer, but neither of these studies produces any spatial output.
The method lineage is weak supervision from clinical metadata. WESEG trains on TCGA slides from pathologists' "percent_tumor_cells" estimates. It marks the top $p\%$ of tiles positive, with tolerance margins for rounding, which assumes area is proportional to the label. Yamaguchi et al. supervise endoscopy frames with a patient-level maximum severity and a loss that penalizes any frame scored above it, which assumes the label is the maximum over frames. Our measurement layer generalizes both. Proportionality and the maximum are special cases of the link function, so we can test whether the extra structure is actually needed. The anchors come from Agarwal et al., who turn RECIST lines into pseudo-masks and reach 90.3% Dice on DeepLesion, and from the healthy-versus-diseased contrast used by PathoGAN. Neither of these uses non-imaging measurements. Text-guided methods such as Gao et al. add language but cannot encode the transport of a biomarker. Approaches that clean labels as noise, such as active label cleaning, treat discordance as random error to be removed. This is the design our proposal argues against.
The Measurement Model
For patient $n$, let $s_\theta(x_{n,i}) \in [0,1]$ be the network's lesion probability at voxel $i$, $v_i$ the voxel volume, $k(i)$ the anatomical compartment containing the voxel, and $z_n$ the vector of observed inflammation covariates (CRP, ADA, neutrophil fraction). The predicted measurement is
$$\hat y_n = a \log\Big(1 + \sum_i b_{k(i)}\, s_\theta(x_{n,i})\, v_i\Big) + c^\top z_n + d .$$
Here $a$ scales the signal and the logarithm encodes saturating dilution. The coefficient $b_k \ge 0$ is the fraction of compartment $k$'s output that reaches the measured fluid; it is near zero for compartments behind barriers when serum is measured. The term $c^\top z_n$ is the inflammatory leakage that Trapé et al. gate on, and $d$ is the baseline. When both serum and local fluid are measured, two such heads share $s_\theta$ and their ratio replaces the fixed 1.2 cutoff.
Training maximizes a Gaussian likelihood of the observed $y_n$, with per-patient variance learned so that genuinely unexplained cases are down-weighted rather than forced to fit. On its own this is not identifiable: scaling $s_\theta$ up and $a$ down gives the same $\hat y_n$. Three anchors resolve this. Healthy scans must have $s_\theta \approx 0$, which fixes $d$ and $c$ given $z$. RECIST lines fix the lesion extent through a pseudo-mask loss in the style of Agarwal et al. An optional handful of full masks per compartment fixes $b_k$. We will report which anchor combinations are necessary.
Research Program
The work runs in three tiers. Each is chosen so its result can be checked.
The first tier is controlled simulation with known ground truth. On BraTS 2017 and the 1,000 hand-segmented DeepLesion lesions, we generate synthetic patient-level "blood" values from the true masks through channels of increasing realism: proportional; saturating; saturating with compartment barriers, assigned per organ region; and saturating with barriers plus a confounder drawn from a synthetic inflammation variable that is partially observed. Because the generating channel is known, this tier answers the core questions directly. It shows how much proportional and max-based methods lose under each channel, whether the learned channel parameters converge to the true ones, and which anchors are needed for that.
The second tier uses a real aggregate label with no paired biofluid. On the TCGA pan-cancer cohort used by WESEG, we replace the fixed top-$p\%$ rule with our learned saturating link plus per-slide slack. This asks whether pathologists' percentage estimates relate to tile area in a non-proportional way that a learned link can exploit. If WESEG's proportional assumption already fits, this tier shows that and caps our claims for histology.
The third tier is a real paired cohort, which the file shows does not yet exist publicly and which we would assemble retrospectively. It consists of abdominal CT from patients undergoing paracentesis, with paired ascitic and serum tumor markers and inflammation markers as in Trapé et al. A small expert-segmented evaluation set covers peritoneal implants. This is the only tier where the full model, with two fluid heads and an inflammation term, is tested on real physiology. Its size will be modest, and we will say so.
Baselines
The naive reference point is a segmentation network trained on RECIST pseudo-masks alone, following Agarwal et al., with the blood value ignored entirely. It reports 90.3% mean Dice on DeepLesion's 1,000 manual masks. Note that this number measures morphological lesion extent. Our claim is not that we beat it on that metric with RECIST present. Our claims are that adding the measurement head does not hurt it, and that the head helps where RECIST lines are missing: Yan et al. note in DeepLesion that unbookmarked lesions are common. To test the second claim, we remove RECIST lines from a controlled fraction of lesions and measure recall on those lesions.
The strongest competitors for supervision from aggregate labels are two. The first is WESEG's proportional top-$p\%$ assignment, which reports an out-of-distribution AUC of 0.932 on TCGA in WESEG. The second is the max-severity loss of Yamaguchi et al., reported at 71.4% accuracy on their private target domain; we reimplement it for continuous labels as a hinge on the maximum. For label-free segmentation on BraTS, PathoGAN reports roughly 72.9% Dice against 86.3% for fully supervised MDGRU in Andermatt et al.. For the paired-fluid setting there is no reported result found on Lacuna for image segmentation supervised by fluid markers. The comparison point there is Trapé's static rule applied as a patient-level gate on training.
All methods share the same backbone, training data, anchor masks and number of training steps, plus an equal hyperparameter search budget of twenty configurations each, selected on the same validation split. On the simulation tier, beating the strongest baseline means a Dice gain that holds across five seeds under the barrier and confounder channels, together with no loss under the proportional channel. On TCGA, it means AUC above 0.932 on the same out-of-distribution split. If we tie, we will report that as evidence that proportionality suffices there.
Two ablations isolate the key ingredient. The first freezes $b_k=1$ and $c=0$, which removes the barrier and confounder terms and leaves only saturating dilution. The second keeps the full parametric form but drops the healthy-scan anchor, which tests whether identifiability, rather than model flexibility, is what helps.
Why not just…?
Why not just treat discordant labels as noise and use a robust loss or label cleaning? Robust losses down-weight patients whose number disagrees with the image. Under compartment barriers, though, the disagreement is systematic: every lesion behind a barrier is under-reported. A noise model will then learn to under-segment those regions instead of ignoring them. The simulation tier tests this directly. We run a robust-loss variant (learned per-patient variance with $b_k=1$, $c=0$) and measure Dice separately inside and outside barrier compartments. If it matches the full model there, the transport structure is unnecessary and we will say so.
Why not just feed the biomarker into the network as an extra input? Conditioning tells the network the patient's systemic state, but the network still needs a training signal for where the lesion is. Concatenating a scalar to the input supplies no spatial supervision, and it invites the network to predict from the scalar while ignoring the image. That shortcut is the same failure that end-to-end prediction from blood shows, given that Oosting et al. found plasma VEGF-A uninformative about lesions. We include this late-fusion variant trained against the same weak labels and expect it to fit $y_n$ well while producing poor masks. The gap between fit and Dice is the diagnostic.
Why not just filter to concordant, low-inflammation patients, as Trapé et al. do, and train a standard model on them? This is the most serious threat. Filtering is simple, and when the confounder is the only problem, filtering may match our model. Its cost is that it discards the inflamed and barrier-affected patients, and the model then has never seen them at test time. We evaluate filtering on all patients and separately on the excluded subgroup. The proposal survives only if the measurement model beats filtering on the excluded subgroup without losing on the retained one. If filtering wins overall, the honest conclusion is that a clinical gate is enough, and our contribution shrinks to the barrier term.
Decisive Checks
Three results would decide the proposal. First, in simulation under barrier and confounder channels, the measurement model must clearly outperform proportional, max-based and filtered training in Dice, and the learned $b_k$ must correlate with the true barrier coefficients. If the parameters do not converge, the model is merely flexible, not identified. Second, on TCGA, the learned link must either exceed WESEG's 0.932 out-of-distribution AUC or collapse to near-proportional, and either outcome is informative. Third, on the assembled ascites cohort, the inflammation coefficient $c$ must come out positive and must absorb the benign high-marker cases, the pancreatitis-like effusions, without the network painting lesions into them. Counting false-positive lesion voxels in benign effusions with high marker values is the most direct test of the whole idea.
References
- 89Zr-Bevacizumab PET Visualizes Heterogeneous Tracer Accumulation in Tumor Lesions of Renal Cell Carcinoma Patients and Differential Effects of Antiangiogenic Treatment
- Comparative Assessment of Two Strategies for Interpreting Tumor Markers in Ascitic Effusions
- Weakly supervised pan-cancer segmentation tool
- Domain Adaptation for Ulcerative Colitis Severity Estimation Using Patient-Level Diagnoses
- Weakly Supervised Lesion Co-Segmentation on Ct Scans
- Pathology Segmentation Using Distributional Differences to Images of Healthy Origin
- Multimodal Weakly Supervised Segmentation for Histopathology
- Active Label Cleaning for Reliable Detection of Electron Dense Deposits in Transmission Electron Microscopy Images
- DeepLesion: automated mining of large-scale lesion annotations and universal lesion detection with deep learning