Radiologists' Ruler Marks as a Utility Signal for CT Denoisers

Image denoisers for medical scans are tuned and judged by how close their output looks to a clean reference. A growing body of evidence says this is the wrong target when the cleaned image will be read by another neural network rather than by a person. Speech researchers found that perceptually better enhancement can leave recognition error unchanged or even make it worse. Medical imaging researchers found that realism scores such as FID barely track how well a segmentation network performs on translated scans. The obvious fix is to score a denoiser by the downstream task directly. But that requires dense, pixel-traced expert masks, and those are the scarce resource that weak supervision exists to avoid. So the situations where a denoiser most needs a task-based check are the ones where no such check can be run.

We propose to close this loop with an annotation hospitals already produce for free. When reporting a tumor, radiologists draw two perpendicular lines across it, its long and short axes. These are RECIST measurements (Response Evaluation Criteria in Solid Tumors), and tens of thousands of them exist in the NIH DeepLesion collection. Our claim is that agreement between a downstream lesion segmentation and these ruler marks is a cheap, label-free-in-practice measure of a denoiser's machine utility. We further claim that the same agreement, made differentiable, can fine-tune a denoiser so that it stops erasing the boundaries a downstream model needs. In two sentences: the weak labels collected to avoid dense annotation can also tell us whether image cleanup helps or hurts the machine reader. We test whether they rank denoisers as dense masks would, and whether training against them yields denoisers that help segmentation more than perceptually tuned ones do.

Core Claim

The research file frames the problem in two directions. On one side is the perceptual–machine gap. Assessment of correlation between objective measures and speech recognition performance and the CHiME-4 distant recognition study show that signal-quality metrics can be uncorrelated or negatively correlated with recognition accuracy. Bridging the Gap reports that enhancement artifacts can push recognition below the raw noisy input. In medicine, Rethinking Perceptual Metrics for Medical Image Translation finds near-zero rank correlation between FID and bone segmentation accuracy, with pixel-level sliced Wasserstein distance as a partial exception. These papers motivate the proposal, but each diagnoses the gap only by evaluating against dense masks.

On the other side, weak supervision shows that RECIST lines carry enough geometry to segment lesions. Weakly Supervised Lesion Co-Segmentation on CT Scans seeds GrabCut from RECIST and trains a co-segmentation network, reaching 90.3% mean Dice on DeepLesion. Those results assume clean, routine-dose images.

The closest work joining the two is ALDEN. It guides a low-dose CT denoiser with frozen DINOv2 and MedSAM features and then checks downstream utility with a fully supervised TotalSegmentator model. ALDEN uses foundation features as a stand-in for "anatomy," and needs dense masks to confirm the result. We replace both: the guidance signal and the evaluation signal are routine clinical measurements tied to the actual lesions of interest. The specific new claim is that RECIST agreement predicts the dense-mask Dice ranking of denoisers better than PSNR, LPIPS, sliced Wasserstein distance, or foundation-feature distance. A second claim is that optimizing it improves segmentation as judged by an independent observer.

Research Program

The first component is a measurement study. We build a set of roughly a dozen denoisers spanning classical filters, CNN regressors trained with pixel losses, GAN-based and diffusion-based restorers, and ALDEN-style feature-guided models. We train them on paired low-dose and normal-dose scans from Mayo2016.

We then corrupt DeepLesion slices with low-dose noise at several levels. DeepLesion ships images rather than raw projection data, so noise insertion must happen in the image domain. We will calibrate its noise power spectrum against Mayo2016 low-dose scans and report results separately for each noise model, since this approximation is a real limitation.

Each denoiser's output is passed to a fixed weakly supervised lesion segmenter, a re-implementation of the RECIST co-segmentation pipeline trained on routine-dose images. We score each output in two ways. The first is RECIST agreement: how well the predicted mask's extents along each radiologist line match the line's length and endpoints, computed over all bookmarked lesions. The second, as ground truth, is Dice against DeepLesion's 1,000 manual segmentations. The main quantity is the rank correlation, across denoiser–noise-level pairs, between each candidate utility metric and true Dice. We also run a second testbed on TotalSegmentator. There we generate pseudo-RECIST lines from dense masks, which lets us test the metric across many structures. Because synthetic lines are cleaner than real radiologist marks, the DeepLesion result remains primary.

The second component turns the metric into a training signal. For a soft predicted mask $m$ and a RECIST line from endpoint $a$ to endpoint $b$, we sample $m$ along the line and slightly beyond each endpoint. The loss rewards high mask probability on the interior of the segment and low probability just outside the endpoints:

$$ \mathcal{L}_{\text{RECIST}} = \frac{1}{|S_{\text{in}}|}\sum_{s \in S_{\text{in}}} \big(1 - m(s)\big) + \frac{1}{|S_{\text{out}}|}\sum_{s \in S_{\text{out}}} m(s), $$

where $S_{\text{in}}$ is a set of points sampled along the segment between $a$ and $b$, and $S_{\text{out}}$ is a set of points sampled a few pixels past each endpoint along the line's direction. We fine-tune a pretrained denoiser $D$ by minimizing pixel fidelity on Mayo2016 pairs plus $\lambda \mathcal{L}_{\text{RECIST}}$, computed through a frozen segmenter on noised DeepLesion slices. The weight $\lambda$ is chosen on a validation split. This loss acts only at the boundaries radiologists marked, which is where over-smoothing does the most damage to edge-driven seed propagation. It also directly tests the file's Gap 1: whether denoising disrupts how GrabCut-style label expansion works.

The obvious failure mode is that the denoiser learns to fool the one frozen segmenter it trains against. To catch this, every evaluation uses a held-out observer: a differently initialized segmenter with a different architecture, trained independently, plus the dense masks.

Baselines

The naive reference point is to apply no denoising. The weakly supervised segmenter runs directly on noised DeepLesion slices, and we also report its clean-image upper bound. The co-segmentation paper reports 90.3% mean Dice on clean DeepLesion. We found no reported result on Lacuna for weakly supervised RECIST segmentation on low-dose-corrupted DeepLesion, so we will establish both numbers ourselves with identical splits.

The strongest existing denoising method is ALDEN, which aligns denoising with anatomy through frozen foundation features. Its paper reports 33.71 dB PSNR on Mayo2016 and a downstream TotalSegmentator Dice 1.07% above the next best method. We will retrain it on the same Mayo2016 pairs, with the same backbone size and the same GPU-hour and hyperparameter-search budget as our fine-tuned denoiser. Beating it means higher held-out-observer Dice on DeepLesion's dense masks at matched or better PSNR, with the gain exceeding the variation across three seeds.

The strongest competitor in noise robustness is a segmenter retrained on noised images with noise augmentation and no front end, given the same compute.

On the metric side, the competitors are the label-free measures already in use. These are PSNR, LPIPS, FID, sliced Wasserstein distance (the partial exception noted in the perceptual-metrics study), and DINOv2 feature distance to the reference. They are scored by rank correlation with true Dice, using bootstrap confidence intervals over denoisers.

The ablation that isolates our key ingredient replaces $\mathcal{L}_{\text{RECIST}}$ with an equally weighted loss that pushes the denoised output's segmentation toward the segmentation of the clean image. That loss is also task-aware but uses no radiologist marks. If it matches our method, the ruler marks add nothing beyond paired data.

Why not just…?

Why not just train the segmenter on noisy images with augmentation and skip denoising? This is the real threat, and the file's evidence against it is weak. The laryngeal-cancer study (A Pipeline to Evaluate the Effects of Noise) shows random-forest accuracy falling from 81.2% to 63.8% under noise. But that classifier used hand-crafted features and had no noise augmentation. A CNN trained with augmentation may absorb low-dose noise well. We run this baseline at full strength. If it matches the fine-tuned denoiser plus fixed segmenter, the honest conclusion is that the front end should go. The metric study still matters in that case, because radiologists view the denoised images, and hospitals need to know which denoisers in that viewing pipeline are safe for downstream tools.

Why not just validate denoisers on a small dense-mask set? DeepLesion already has 1,000 manual masks. If 100 of them rank denoisers as reliably as 30,000 RECIST marks, the proxy is unnecessary. We test this directly by subsampling dense masks and comparing the stability of each ranking. The proxy is only worth recommending if it reaches comparable rank agreement where dense masks are absent or too few. That case covers lesion types and sites outside the annotated subset.

Why not just use a label-free proxy such as sliced Wasserstein distance or foundation-feature distance? These require no annotation at all, and ALDEN shows foundation features can steer denoising. However, Rethinking Perceptual Metrics found that feature-based metrics track texture more than geometry. Bring the Noise reports that pretrained extractors are brittle under input noise. The correlation experiment compares them head to head. If DINOv2 distance correlates with Dice as well as RECIST agreement does, we will report that the free proxy suffices.

Proposed Experiments

The first decisive check is the rank correlation between RECIST agreement and true dense-mask Dice across all denoiser–noise pairs on DeepLesion. It is compared against PSNR, LPIPS, FID, sliced Wasserstein distance and DINOv2 distance, with bootstrap intervals. The claim fails if RECIST agreement is not clearly above the best label-free metric.

The second check is the fine-tuning result: held-out-observer Dice on the 1,000 masks, and the 95th-percentile Hausdorff distance. Hausdorff distance measures the worst boundary errors, which is where over-smoothing should show. These are compared against no denoising, ALDEN, the noise-augmented segmenter, and the clean-segmentation-consistency ablation. We also report Mayo2016 PSNR to show what perceptual fidelity is traded away.

The third check targets adversarial overfitting. If Dice gains appear for the frozen training segmenter but vanish for the held-out observer, the method is exploiting its observer rather than restoring boundaries, and we will report it as such.

Finally, the TotalSegmentator pseudo-RECIST testbed tests whether the metric's predictive power extends from lesions to normal anatomy. It also lets us compare directly with the downstream setting ALDEN used.