Does Compressing 3D Scans Hurt Rare Findings Most?

Models that read 3D medical scans, such as chest CTs, face a cost problem. A single volume becomes more than ten thousand visual tokens, the small image pieces a language model reads. So practical systems throw most of them away, either by subsampling slices or by pruning tokens that look redundant. Published results say this is nearly free. CARVE keeps 20% of tokens and retains 98.1% of full-token accuracy across 3D visual question-answering benchmarks.

That figure is an average, and averages are dominated by common cases. This note argues that the loss from compression is not spread evenly. It should fall mostly on findings that are both rare in the training data and confined to a few slices. A small nodule on one or two slices is exactly the "redundant-looking" evidence a pruner discards. The model's weak prior for that finding also means that whatever evidence survives is less likely to be used. If this holds, compression quietly converts a data-imbalance problem into a spatial-context problem, and headline retention numbers hide it.

We propose to measure this effect properly, which requires statistics built for small groups. We then test whether a cheap change to how the token budget is spent removes it.

Core Claim

The prediction is specific. The accuracy a finding loses when volumetric context is compressed grows as its training prevalence falls and as its slice extent shrinks. At a matched token budget, the gap between common and rare findings is therefore wider after compression than before.

The fix we test is to reserve a small share of the token budget for depth windows where a cheap zero-shot scorer detects evidence for any finding in a fixed vocabulary. That scorer's votes are weighted by inverse prevalence, so rare findings get guaranteed slots rather than competing with common anatomy for slots judged by redundancy.

The closest work, CARVE, allocates budget by redundancy between slices and never reports results broken down by finding frequency. We aim to show both that the frequency-dependent loss exists and that a prevalence-aware reservation removes it where redundancy-based retrieval does not.

Why This Intersection

The volumetric literature supplies the mechanism. Roth et al. segmented the pancreas slice by slice and showed that ignoring continuity between slices costs a great deal: 3D smoothing raised Dice overlap from 56% to 68%. This motivates treating depth context as fragile.

CT-CLIP and CT-CHAT supply the model we will compress. CT-CLIP is a native 3D encoder trained on reports from the 50,188-volume CT-RATE dataset, and CT-CHAT pairs it with a language model. CT-CLIP reaches 0.731 mean zero-shot AUROC and generalizes externally to RAD-ChestCT.

fVLM motivates the concern that local findings get averaged away. It argues that whole-volume contrastive learning dilutes small findings into whole-chest embeddings, and it fixes this with anatomy-level alignment. We keep that diagnosis but move it from pretraining to inference-time compression.

The imbalance literature supplies the regime and the evaluation discipline. Larrazabal et al. showed on 2D chest X-rays that underrepresented groups lose accuracy. We ask whether the depth axis adds a second, compression-driven penalty on top of that. The standard-error argument from the imbalance direction matters here:

$$SE = \sqrt{\sigma_1^2/n_1 + \sigma_2^2/n_2}$$

In this expression, $n_1$ and $n_2$ are the numbers of cases in two groups and $\sigma_1^2, \sigma_2^2$ are their variances. It implies that per-finding metrics for rare labels are dominated by the small positive count. Averaged retention scores can therefore neither detect nor rule out a rare-finding penalty.

Park et al. constrains our design. Their joint model on an imbalanced cohort of 671 controls and 34 cases showed that preprocessing done independently of the minority signal strips that signal away. Compression is exactly such a signal-blind preprocessing step. Liu et al. showed that naive weighting collapsed a minority group's effective sample size to 155, and it recovered to 1,105 under overlap weighting. That lesson shapes our analysis: we report effective positive counts and avoid weighting schemes that let a handful of cases dominate.

Research Program

The first part is a measurement study. We take CT-CLIP for zero-shot classification of the 18 CT-RATE abnormality labels, and CT-CHAT for question answering and report generation. We insert each compression method between the 3D encoder and the downstream head, sweeping keep ratios from 5% to 100%.

For each finding we record two properties. The first is training prevalence, computed from CT-RATE labels; the file does not list per-label prevalences, so we compute them ourselves. The second is slice extent: the number of depth windows whose occlusion drops the full-token model's score for that finding by more than a fixed fraction.

The main quantity is the compression-induced gap for finding $k$ at keep ratio $r$:

$$G_k(r) = \mathrm{AUROC}_k(1) - \mathrm{AUROC}_k(r)$$

This separates compression's damage from the scarcity penalty already present at full tokens. We fit $G_k(r)$ against log-prevalence and slice extent using a mixed model with finding and volume effects. We report patient-level bootstrap confidence intervals and the positive count behind every per-finding number. Findings with too few test positives are pooled into prevalence bins rather than reported alone, so wide intervals are not mistaken for "no effect."

The second part is the intervention. A reserved fraction $\rho$ of the budget, around 5% of total tokens, goes to depth windows ranked by

$$s_w = \sum_k \pi_k^{-1/2} p_k(w)$$

Here $p_k(w)$ is CT-CLIP's zero-shot score for finding $k$ on window $w$, and $\pi_k$ is that finding's training prevalence. The square-root tempering avoids the extreme weights that Liu et al. warn about. The remaining budget is spent by CARVE unchanged.

The reservation needs no retraining and uses the same encoder the system already runs. Its only extra cost is one pooled score per window.

Baselines

The naive reference is uniform depth subsampling at the matched token budget. It is what most memory-limited pipelines do. We run it with the same encoder, head, and prompts.

The full-token model is the upper reference that defines $G_k$. It is not a competitor.

The strongest existing method is CARVE. It combines adaptive windowing, anchors within each slice, retrieval of residual tokens across slices, and local token folding. Its reported result is 98.1% of full-token accuracy retained at a 20% keep ratio across AMOS-MM, 3D-RAD, and M3D-VQA. It is the strongest competitor because its residual retrieval is explicitly meant to rescue evidence confined to one or two slices, which is precisely our concern.

We will run CARVE at identical keep ratios, with its settings tuned on the same validation split and given the same search budget as our reservation fraction $\rho$. For zero-shot classification with CT-CLIP on CT-RATE, the reference is 0.731 mean AUROC (Hamamci et al.). No per-label or compressed-token result for CT-CLIP was found on Lacuna.

A generic saliency baseline keeps tokens by the encoder's attention or norm. It tests whether any content-aware selection suffices.

Beating CARVE means a statistically reliable reduction in the rare-bin gap $G_k(0.2)$ at matched budget, with the common-bin gap and mean retention no worse than CARVE's by more than a pre-registered margin.

The key ablation replaces $\pi_k^{-1/2}$ with 1. This keeps the reserved budget and scorer but removes prevalence awareness, and it isolates whether rarity weighting, rather than extra evidence-driven slots, does the work.

Why not just…?

Why not just keep more tokens? If the rare-finding penalty disappears at a 40% keep ratio, the problem is a budget choice, not a research question. We will report the keep ratio at which each prevalence bin recovers. The claim only stands if rare bins need substantially more budget than common ones, which would mean a uniform budget increase buys rare-finding accuracy expensively. The reservation should then match that recovery at a lower budget.

Why not just rebalance the downstream head with class-weighted losses or oversampling? Reweighting changes how surviving evidence is used; it cannot restore tokens already deleted. We test this directly by fine-tuning a class-balanced linear probe on compressed CT-CLIP features. If the balanced probe closes the gap, compression is not the cause and the proposal fails. We will say so.

Why not just trust CARVE's residual retrieval, which already targets one-slice evidence? This is the real threat. Residual retrieval may already catch small lesions regardless of how common they are. If CARVE's $G_k$ shows no dependence on prevalence, the measurement study still stands as a negative result with proper small-group statistics, and the intervention is dropped. The ablation with rarity weighting removed tells us whether any remaining gain comes from rarity or merely from extra slots.

Why not just use a 2D slice model with smoothing and no compression? Wolf et al. found that 2D vision-language models sit near chance on relative-position questions in CT slices. Hamamci et al. found that 2D slice-based assistants trail the native 3D CT-CHAT in radiologist ratings. So a 2D model gives up the depth context we are trying to protect.

Proposed Experiments

The primary setting is CT-RATE zero-shot classification, with RAD-ChestCT as an external check that the prevalence dependence is not specific to one scanner protocol. Question answering and report generation are evaluated on AMOS-MM, M3D-VQA, and 3D-RAD. We tag each question by the finding it concerns, and then by that finding's frequency among the questions.

The decisive check comes in two parts. First, under uniform subsampling and CARVE, the slope of $G_k(0.2)$ on log-prevalence must be negative with a confidence interval excluding zero, after controlling for slice extent. Second, the prevalence-aware reservation must flatten that slope relative to both CARVE and the ablation with rarity weighting removed, at identical token counts.

A secondary check tests the mechanism. The damage should concentrate in findings with small slice extent. Occlusion analysis should show that the rescued windows are those the full-token model relied on.

We will pre-register the prevalence bins, the retention margin, and the minimum positive count per bin. With only a few dozen positives, an absent effect in the rare bins must not be read as evidence of no effect, in line with the variance argument above.