Radiologists already mark tumors as part of routine care. They draw two perpendicular lines across a lesion, the longest diameter and its cross-axis, following the RECIST convention (Response Evaluation Criteria in Solid Tumors). They almost never trace full 3D outlines. This note proposes to use those two lines directly to predict outcomes. A frozen pretrained image encoder produces a grid of local features. We average those features over a soft region defined by the calipers, combine the result with a few clinical variables, and fit a low-capacity survival or risk model. No segmentation network is trained at any point. The claim is that for cohorts under about 100 patients, the calipers help mainly by stopping the image model from looking at the wrong places, not by producing an accurate mask. If that is right, a cheap caliper-defined region should match full expert masks for prediction and clearly beat both whole-image features and the caliper length used as a single number.
This matters because two literatures that should meet do not. Weak-supervision work shows that calipers carry enough geometry to train segmenters close to full-mask quality, and then stops at the mask. Multimodal prognosis work shows that lesion features fused with clinical tables improve risk prediction, but it assumes dense 3D masks and often large cohorts. A hospital studying a rare tumor has the calipers and 50 patients. It has neither the masks nor the 5,000 patients.
Core Claim
In small cohorts, what the image branch lacks is a limit on what it can fit, not raw information. Prior evidence of the failure is stark. Jose et al. segmented BraTS gliomas with full supervision and fed shape and clinical features to a small neural network. Survival accuracy fell from 67.9% (Spearman 0.496) on validation to 46.8% (Spearman 0.148) on test, with 163 training cases. Bria et al. note that many clinical AI cohorts have fewer than 100 subjects. An investigation of chest X-ray models shows that such models latch onto scanner and site artifacts.
We hypothesize the following. Pooling frozen features over a caliper-defined region removes most of these degrees of freedom. Treating the caliper geometry as a range of possible regions, rather than a single pseudo-mask, keeps boundary errors from turning into prediction errors.
Concretely, following Zhou et al., we use two shapes built from the caliper endpoints:
- The quadrilateral $Q$ joining the endpoints. It is almost surely inside the lesion.
- The minimum bounding circle $C$. It almost surely contains the lesion.
On the key slice, and propagated to neighboring slices by shrinking with distance, each image location $x$ gets a weight $w(x)$. The weight is 1 inside $Q$, 0 outside $C$, and falls off smoothly in the band between them. The lesion descriptor is
$$z = \frac{\sum_x w(x) F(x)}{\sum_x w(x)},$$
where $F(x)$ is the frozen encoder's feature at location $x$. We also pool a second descriptor over the ring just outside $C$ to capture the tissue surrounding the tumor. The descriptors are reduced by PCA fitted within each training fold and concatenated with clinical variables. A ridge-penalized Cox model, or logistic model, makes the prediction.
The whole pipeline has no trained image parameters. It runs in minutes on a CPU. The contrast in cost with the 400–525 GPU hours reported for CLIMATv2 is deliberate.
Relation to Prior Work
Two papers supply the geometric machinery, and we reuse it differently.
- Agarwal et al. turn DeepLesion RECIST lines into GrabCut pseudo-masks and train a co-segmentation network, reaching 90.3% Dice.
- Zhou et al. train two networks under a consistency loss confined to the band between $Q$ and $C$, reaching 0.907 Dice on KiTS19 against a 0.912 full-supervision ceiling.
Both treat the mask as the goal and need hundreds to thousands of scans to train the segmenter. We never train a segmenter, and we ask only whether the region helps prediction.
Three papers motivate the gap but work outside our regime.
- MFM-Geom fuses bp-MRI and clinical-text tokens through a covariance-matrix head and does well with 10% of PI-CAI. It relies on centered crops and gives no route from weak annotations.
- PROGPATH fuses slide images with clinical variables for survival without tile labels, but trains on 7,999 slides.
- CLIMATv2 models interactions between specialist "agents" on cohorts of thousands.
We treat all three as evidence that interactive fusion helps at scale. We do not assume it helps below 100 patients, and we test that directly.
Two papers constrain the design.
- Limitations of Synthetic Data Generation finds that GAN and diffusion augmentation in specialized scarce domains memorizes training cases or produces idealized examples, and does no better than AugMix. We therefore avoid generative components, including DiffRect-style diffusion rectification of pseudo-labels (DiffRect).
- The MIMIC-IV-MM study (Multimodal Risk Prediction) finds late fusion "particularly robust." We therefore test simple fusion before any interactive fusion.
SpurCon removes shortcuts using metadata-anchored contrastive training. We test whether spatial restriction alone achieves a similar protection with no training at all.
Research Program
The work has three parts.
Build the caliper simulator and pooling pipeline. Where expert masks exist, simulate calipers from them. On the slice of largest lesion area, take the longest in-plane diameter and its longest perpendicular. We also add realistic endpoint jitter so that the calipers do not inherit the precision of the masks.
Run a controlled comparison at matched sample size. Hold the frozen encoder and the prediction head fixed. Vary only how image features are chosen:
- whole image,
- a fixed crop centered on the caliper midpoint,
- the diameter as a scalar,
- a hard $Q$ mask,
- a hard $C$ mask,
- the soft $Q$–$C$ weighting,
- the soft weighting plus the surrounding ring,
- the full expert mask.
We run this across training sizes of 25, 50, 100, and all available patients.
Test the mechanism. If the benefit comes from restricting what the model can see, it should grow as $N$ shrinks. It should survive caliper noise. It should also resist a planted spurious signal outside the lesion.
For encoders we use the BiomedCLIP image tower as adapted in MFM-Geom, applied slice by slice. An ImageNet ResNet serves as a control. This lets us check whether the effect depends on domain pretraining, a dependence that TIMNet documents for classification.
Data and evaluation. The primary benchmark is BraTS 2018 survival: 163 cases with multi-sequence MRI, full masks, and age. This makes it the one dataset in the file with small $N$, dense masks for simulating calipers, and an outcome. PI-CAI with PROSTATE158 as the external set is a secondary benchmark, with PSA, volume, and age as clinical variables. It depends on whether lesion delineations are available for the positive cases; if they are not, the PI-CAI arm uses only crop-based variants.
All results use repeated nested cross-validation with 20 repeats of 5 folds, with PCA and penalty tuned in the inner loop. This is because Jose et al. illustrate how misleading a single split can be at this size. Metrics are Harrell's C-index, Spearman correlation with survival days, and the BraTS three-class survival accuracy (short, mid, long) so that results can be compared with the reported numbers. Calibration is measured by integrated Brier score. For PI-CAI we use AUC-PR and false-positive rate at 95% sensitivity.
Baselines
Clinical-only reference. A ridge Cox model on age alone for BraTS, and on PSA, volume, and age for PI-CAI. This shows what the images add at all. TSB shows that a local Cox model on under 200 patients can sit at C-index 0.500, so this bar may be low. It still has to be measured on our own splits.
Strongest published pipeline on BraTS. The two-stage system of Jose et al., a fully supervised segmenter followed by shape, fractal, and clinical features in a small neural network. It reports 67.9% accuracy and Spearman 0.496 on validation, and 46.8% accuracy and Spearman 0.148 on test. It is the natural competitor because it uses more annotation than we do, namely full masks. We cannot rerun its challenge test split. Instead we re-implement its feature set on the expert masks and feed it to the same ridge head and the same nested cross-validation, which isolates the effect of the features from the effect of the head. Beating it means our caliper pipeline matches or exceeds the re-implementation's cross-validated Spearman and C-index. We also report where both sit relative to the published 0.148 test figure, without claiming a like-for-like comparison.
Strongest weak-supervision competitor. A segmenter trained with the $Q$/$C$ co-training of Zhou et al. on the same simulated calipers, whose predicted masks then feed the same pooling and head. It gets a fixed GPU budget and the same folds, with segmenter training kept inside each training fold. Our claim is that the training-free soft region is at least as good for prediction when $N \le 100$. If the trained segmenter wins by a margin whose confidence interval excludes zero, that claim fails.
Strongest fusion method in the file. On PI-CAI, MFM-Geom, which reports AUC-PR 90.67 with 10% of the training data. We run it at our subsample sizes with its published architecture, tuned on the inner folds with the same number of configurations as our head. It is fed either centered crops or our caliper-pooled tokens. This separates two questions: whether the covariance head helps, and whether the caliper region helps.
Ablations isolating the key ingredient. Three comparisons do this.
- Diameter-as-scalar plus clinical tests whether the gain is merely tumor size.
- Hard $Q$ or $C$ masks versus the soft weighting tests whether treating geometry as a range of regions matters.
- Soft weighting versus the full expert mask gives the ceiling.
Late fusion, with separate image and clinical models whose risk scores are averaged, is compared against concatenation in every configuration.
Why not just…?
Why not just use the caliper length and clinical variables? RECIST diameter is already a strong prognostic quantity, and a ridge Cox model on it is almost impossible to overfit. This is the most serious threat. If pooled features add nothing over the scalar at $N \le 100$, the honest conclusion is that calipers are useful as a number, not as a region. The diameter-as-scalar ablation reports exactly this. We will report it even if it wins.
Why not just feed a crop or the whole image to a frozen foundation model? Pretrained features plus a penalized head are already low-capacity, and TIMNet shows that domain pretraining carries much of the load with little labeled data. Our mechanism predicts that crops dilute lesion features with background and admit site artifacts, so the gap should widen as $N$ shrinks. The planted-artifact test decides this. We insert a faint marker outside the lesion that correlates with outcome in the training folds but not in the test folds. We expect whole-image and crop pooling to degrade on test while caliper pooling should not. If crops match caliper pooling on both clean and planted data, the region adds nothing and we say so.
Why not just train a weak segmenter first, since it reaches about 0.9 Dice? The results of Zhou et al. and Agarwal et al. come from 210 to more than 32,000 scans. Whether they hold at 50 is unknown. Even at good Dice, the errors sit exactly at the boundary, where texture features live, and Jose et al. show that two-stage pipelines overfit at this scale. The trained-segmenter baseline settles this directly.
Why not use interactive cross-attention or covariance-based fusion? These designs win at scale (CLIMATv2) and at 10% of PI-CAI (MFM-Geom). However, late fusion was the robust choice on MIMIC-IV-MM (Multimodal Risk Prediction). The MFM-Geom comparison across subsample sizes shows where, if anywhere, the extra fusion capacity starts paying for itself.
Why not augment the data with generative models? In specialized scarce domains, generation memorizes or idealizes cases and does not beat AugMix (Limitations of Synthetic Data Generation). We include AugMix applied to the image before encoding as a control, and do not pursue generation.
Decisive Checks
The proposal succeeds only if all of the following hold on BraTS under repeated nested cross-validation:
- At $N \le 100$, soft caliper pooling plus clinical variables beats both diameter-plus-clinical and crop-plus-clinical in C-index, with paired bootstrap intervals excluding zero.
- It comes within 0.02 C-index of full-mask pooling.
- It is not beaten by the trained weak segmenter.
- Its advantage over crops is larger at $N = 25$–$50$ than at the full cohort.
- Under planted-artifact and caliper-jitter perturbations, it degrades less than whole-image pooling.
If the first check fails against the scalar, the finding is still useful: the region adds nothing beyond size. If the fourth fails, the capacity-restriction explanation is wrong, even if the method works for other reasons.
References
- Glioma Prognosis: Segmentation of the Tumor and Survival Prediction Using Shape, Geometric and Clinical Information
- Towards AI-driven Next Generation Personalized Healthcare and Well-being
- Unveiling COVID-19 from CHEST X-Ray with Deep Learning: A Hurdles Race with Small Data
- RECIST Weakly Supervised Lesion Segmentation via Label-Space Co-Training
- Clinically-Inspired Multi-Agent Transformers for Disease Trajectory Forecasting From Multimodal Data
- Weakly Supervised Lesion Co-Segmentation on Ct Scans
- A Geometric Multimodal Foundation Model Integrating Bp-MRI and Clinical Reports in Prostate Cancer Classification
- Pancancer outcome prediction via a unified weakly supervised deep learning model
- Limitations of Synthetic Data Generation in Specialized Data-Scarce Domains
- DiffRect: Latent Diffusion Label Rectification for Semi-supervised Medical Image Segmentation
- Multimodal risk prediction with physiological signals, medical images and clinical notes
- SpurCon: Weighted Supervised Contrastive Learning for Mitigating Spurious Cues in Medical Imaging
- Contrastive Cross-Modal Pre-Training: A General Strategy for Small Sample Medical Imaging
- Boosting Transfer Learning with Survival Data from Heterogeneous Domains