Radiologists already mark most tumors they report. They draw two crossing lines, the RECIST diameters (Response Evaluation Criteria in Solid Tumors), on one CT slice. Several groups have turned these lines into full segmentation masks and reported Dice overlap near the fully supervised ceiling. Those results come from curated test sets, where lesions tend to be round, well contrasted, and clearly bounded. The lesions that actually slow a radiologist down are irregular, infiltrative, or low-contrast. On those cases every current pipeline makes a hard yes/no decision about the uncertain rim. One way assumes the tumor is convex. The other runs GrabCut, a graph-cut tool that is brittle in low contrast. This proposal uses spatial fuzzy clustering instead: a clustering method in which each pixel holds a graded membership in "lesion" and "background," and neighboring pixels pull on each other's memberships. We use it to fill in only the uncertain rim between the RECIST lines and their bounding circle. The graded memberships become soft training targets for a segmentation network, and they also become a displayed "edit band" that tells the reader where the boundary is in doubt.
The second half of the proposal is the evaluation. Curated benchmarks overstate performance on ambiguous cases. In stroke imaging, AUC drops from 0.934 to 0.862 once subtle cases are kept in (Kim et al.). We therefore judge the method on a frozen-model, consecutively acquired stream of routine RECIST-annotated scans. We stratify results by lesion irregularity and contrast. We also measure what matters in practice: how long a radiologist takes to reach an acceptable 3D contour.
Core Claim
Image-driven graded memberships, restricted to the band that RECIST geometry cannot resolve, beat both hard pseudo-masks and image-blind soft labels. The gain should concentrate on concave and low-contrast lesions. If true, it would show up exactly where curated splits hide it. On round, clear lesions we expect near-parity with the best existing method.
Zhou et al.'s label-space co-training reaches 0.907 Dice on KiTS19 against a 0.912 fully supervised ceiling (RECIST Weakly Supervised Lesion Segmentation via Label-Space Co-Training). That leaves little headroom on average, so an average-Dice win is not the claim. The claim is about where the remaining error lives, and whether a graded boundary reduces correction work.
Prior Work and Where This Sits
The weak-supervision line supplies the setting and the strongest competitors.
- Agarwal et al. seed GrabCut from RECIST lines and train a Siamese co-segmentation network, reaching 90.3% Dice on DeepLesion (Weakly Supervised Lesion Co-Segmentation on Ct Scans). They report that their quality depends on the initial GrabCut masks.
- Cai et al. propagate GrabCut masks into 3D with self-paced teacher-student training and a stacked GAN. They report 93% Dice on RECIST slices but 76% on full volumes (Accurate Weakly Supervised Deep Lesion Segmentation on CT Scans: Self-Paced 3D Mask Generation from RECIST).
- Zhou et al. replace GrabCut with an inner quadrilateral and an outer circle and co-train inside the margin between them. This assumes convexity.
All three convert the margin into hard labels. Our proposal keeps the Zhou margin as the region of uncertainty but fills it with image evidence and graded memberships.
Two papers constrain the design.
- Agarwal et al. found that dense CRF refinement (a conditional random field, a pixel-smoothing model applied after the network) sharply reduced recall.
- Can et al. train from scribbles with random-walk expansion, CRF relabeling, and dropout-based filtering of uncertain labels (Learning to Segment Medical Images with Scribble-Supervision Alone). They found CRF relabeling fragile to its settings.
Both warn against uniform pairwise smoothing. That is why our spatial term must switch off at boundaries.
The clustering mechanism comes from Liew's adaptive spatial fuzzy c-means (Current Methods in the Automatic Tissue Segmentation of 3D Magnetic Resonance Brain Images). It turns neighborhood smoothing up in homogeneous regions and down where local dissimilarity is high, and it reports under 4% misclassification on BrainWeb with 40% intensity non-uniformity. Kang and Kim's homogeneity-weighted variant adds automatic choice of cluster count (VS-FCM). Both papers ran without supervision on phantoms or 2D test images. Neither was anchored to clinical marks or used to train a network, and that coupling is what we add.
The validation design is motivated by Direction 2. Meng et al. measured reader accuracy, inter-reader kappa (0.34 to 0.58), and a 25.7% drop in decision time in a multivendor reader study (AI support for accurate and fast radiological diagnosis of COVID-19). Ng et al. staged a prospective rollout and showed that assistance can add reading workload (4–6%) even when it helps detection (Prospective implementation of AI-assisted screen reading). Both studied classification. No segmentation-assistance tool trained from weak labels has been evaluated this way. Three further papers argue that curated history hides sequential drift, borderline cases, and workload effects: Zhao et al., Wang et al., and Chao et al..
Research Program
Pseudo-labeler. For each RECIST-marked slice we build the inner quadrilateral $Q$ and the outer circumscribed circle $C$, following Zhou et al. Pixels inside $Q$ are fixed to lesion, and pixels outside a dilated $C$ are fixed to background. Inside the band, we run a two-class fuzzy clustering that minimizes
$$J=\sum_{i\in B}\sum_{k}u_{ik}^{m}\Big(\lVert x_i-v_k\rVert^2+\alpha_i\,\tfrac{1}{|N_i|}\sum_{j\in N_i}\lVert x_j-v_k\rVert^2\Big).$$
The symbols are defined as follows:
- $B$ is the band.
- $u_{ik}\in[0,1]$ is pixel $i$'s membership in class $k$.
- $m>1$ is the fuzziness exponent.
- $x_i$ is a local intensity feature.
- $v_k$ is the class centroid, initialized from the clamped regions.
- $N_i$ is a $3\times3\times3$ neighborhood.
- $\alpha_i=\alpha_0\exp(-d_i/\tau)$ is a spatial weight that decays with Liew's local dissimilarity $d_i$, so smoothing turns off at edges.
Clamping the centroids to the RECIST-certain regions removes the initialization sensitivity that troubles unsupervised fuzzy clustering. The clustering runs slice by slice outward from the RECIST slice, with the previous slice's memberships constraining the next. This gives a 3D extension without a GAN. For MRI scribble experiments we add Liew's smooth bias-field term. For CT we omit it because CT has no comparable gain field.
Training. The network is trained with ordinary cross-entropy on clamped pixels. On band pixels it uses soft cross-entropy against $u_{i,\text{lesion}}$. Network predictions refresh the band memberships once per round, for at most two rounds, so that the network does not learn back its own errors.
Edit band. At inference, pixels whose predicted probability falls between 0.2 and 0.8 are shown as a translucent band around the contour. The reader accepts the contour or edits it.
Baselines
The naive reference point is the RECIST ellipsoid: an ellipse fitted to the two diameters and extruded through adjacent slices. Cai et al. report it gives volume $R^2=0.62$ against manual volumes, versus 0.78 for their method. It costs nothing, and any method must clearly exceed it on 3D Dice and volume agreement.
The strongest existing method is Zhou et al.'s label-space co-training, which reports 0.907 Dice on KiTS19 with a Swin backbone. We will reimplement it on the same backbone, the same RECIST simulations drawn from KiTS19 ground truth, and the same training schedule and hyperparameter budget as ours, sweeping both methods over equal numbers of configurations on a held-out validation split. Beating it means two things. First, matching it within 0.5 Dice overall. Second, a statistically significant improvement, by paired bootstrap over lesions, on the lowest tercile of lesion solidity (area divided by convex-hull area) and on the lowest contrast tercile.
GrabCut-seeded training, as in Agarwal et al. and Cai et al., is the second baseline. Zhou et al. report it at 0.891 on KiTS19, and Agarwal et al. report 90.3% on DeepLesion. We run it within the same network and budget.
SAM prompted with the box spanned by the RECIST endpoints is the zero-shot competitor. Its masks serve both as direct outputs and as hard pseudo-labels for training. We found no reported result on Lacuna for SAM on DeepLesion or KiTS19. The only figure in our material is a roughly 5% margin over prior methods on breast ultrasound (de Oliveira et al.).
Three ablations isolate the key ingredient. The first fills the band with image-blind soft labels from a signed distance transform between $Q$ and $C$. This tests whether any soft margin helps, or only image-driven memberships do. The second thresholds our memberships at 0.5, which tests soft versus hard. The third sets $\alpha_i$ constant, which tests adaptive versus uniform spatial weighting.
Why not just…?
Why not just prompt SAM with the RECIST box? This is the real threat. RECIST endpoints define a box almost for free, and box prompts were SAM's strongest mode in de Oliveira et al. Their evaluation was 2D only, and SAM returns a hard mask with no graded boundary. The SAM baseline above settles it directly. If SAM pseudo-labels match ours on the irregular and low-contrast strata, we will drop our clustering stage. We would then run the fuzzy membership step on top of SAM's mask band, so the soft-target and edit-band claims would be tested on the better initializer.
Why not just use label smoothing or distance-based soft targets? If softness alone helps, the clustering is unnecessary. The distance-transform ablation answers this. It uses the same band and the same soft loss, but ignores the image. Our claim predicts it helps on round lesions and fails on concave ones, because it cannot bend the boundary toward the real edge.
Why not just add a dense CRF after the network? Agarwal et al. found that dense CRF refinement cost substantial recall. Can et al. found CRF relabeling fragile enough to need dropout-based filtering. We include a CRF-refined version of the Zhou baseline so that this objection is measured rather than argued.
Why not just collect full masks? Full masks remain the upper bound. KiTS19 supplies it at 0.912. The question is how much of that gap matters clinically, which the reader study measures in minutes rather than Dice.
Proposed Experiments
The retrospective stage covers three datasets.
- KiTS19. Full masks are available, and we simulate RECIST lines from them, with jitter in endpoint placement.
- DeepLesion. We use real RECIST bookmarks and the 1,000 manually segmented test lesions used by Agarwal et al.
- Lymph-node CT. Cai et al. report 70.2% weak versus 71.0% full supervision there, which is a check on small, low-contrast targets.
A secondary transfer check runs the scribble version, with the bias-field term, on ACDC and the NCI-ISBI 2013 prostate set, against Can et al.'s 88.0% and 78.1%. BrainWeb with injected non-uniformity serves as a controlled stress test of the pseudo-labeler alone.
The decisive retrospective check is the stratified analysis. If the gain over Zhou et al. does not concentrate in the low-solidity and low-contrast terciles, the core claim is false.
The prospective-style stage freezes all models on a fixed date. At a partner site we then collect consecutive post-freeze CT scans that carry routine RECIST marks, with no case exclusion. We report Dice against two independent expert contours, and the inter-expert Dice serves as a noise ceiling.
A crossover reader study then gives six radiologists matched lesions under three conditions, separated by a washout period:
- correcting a hard Zhou-style mask,
- correcting our contour with the edit band shown,
- contouring from the RECIST lines alone.
We record time to an accepted 3D contour, edit volume, and inter-reader agreement. This follows the measurements of Meng et al. We heed Ng et al.'s finding that assistance can add work, so assistance counts as successful only if it reduces total time relative to manual contouring.
The decisive human check has two parts. The edit band must cut correction time on irregular lesions relative to the hard mask, and it must do so without lowering final agreement with expert contours. A drop in agreement would signal that readers trust the band too readily.
References
- Deep learning-based classification of diffusion-weighted imaging-fluid-attenuated inversion recovery mismatch
- RECIST Weakly Supervised Lesion Segmentation via Label-Space Co-Training
- Weakly Supervised Lesion Co-Segmentation on Ct Scans
- Accurate Weakly Supervised Deep Lesion Segmentation on CT Scans: Self-Paced 3D Mask Generation from RECIST
- Learning to Segment Medical Images with Scribble-Supervision Alone
- Current Methods in the Automatic Tissue Segmentation of 3D Magnetic Resonance Brain Images
- VS-FCM: Validity-guided Spatial Fuzzy c-Means Clustering for Image Segmentation
- AI support for accurate and fast radiological diagnosis of COVID-19: an international multicenter, multivendor CT study
- Prospective implementation of AI-assisted screen reading to improve early detection of breast cancer
- What benefit can be obtained from magnetic resonance imaging diagnosis with artificial intelligence in prostate cancer compared with clinical assessments?
- Automated segmentation and diagnosis of pneumothorax on chest X-rays with fully convolutional multi-scale ScSE-DenseNet: a retrospective study
- Foundation versus domain-specific models for left ventricular segmentation on cardiac ultrasound
- Zero-Shot Performance of the Segment Anything Model (SAM) in 2D Medical Imaging: A Comprehensive Evaluation and Practical Guidelines