Models that read a CT or MRI scan and write a report, or answer questions about it, cannot afford to look at the whole scan. A single chest CT yields over ten thousand visual tokens, so systems either thin out the slices or prune tokens down to a fraction of the original. This loses depth information, which we call volumetric context loss. When depth evidence is missing, a language model will fall back on whatever it has read most often. We propose to measure this with an old tool from lexical simplification: corpus frequency, the observation that rare words are harder for humans to process. Here we apply it to how often each finding appears in radiology reports. Our hypothesis is that compression does not harm findings evenly. It mostly erases rare findings, which tend to be small and visible on only one or two slices, while common findings survive because the model would have predicted them without looking. Aggregate scores hide this. We then use the same frequency signal to decide which slices and tokens to keep.
The opportunity is that the strongest current compressor reports near-lossless averages. CARVE keeps 98.1% of full-token VQA performance at an 80% compression rate. Separately, Your other Left! shows that vision-language models answer CT spatial questions from memorized anatomy: GPT-4o's accuracy falls from 75% to 50% when slices are flipped. If those two facts interact, a 98% average could hide a large failure on exactly the findings that matter most clinically. In two sentences: compressed 3D medical models replace missing depth evidence with common report phrases, so their errors concentrate on rare findings. We will show this with frequency-stratified evaluation and fix it by spending the visual token budget where the report-frequency prior is weakest.
Core Claim
Our claim has two parts. The first is diagnostic. Under volumetric compression, the per-finding performance drop of a 3D vision-language model rises with how rarely that finding appears in the training reports. On compressed inputs, outputs move toward what a "blind" copy of the model produces, meaning the same model run with no visual tokens. The second part is a method. A token-retention scorer can learn which visual tokens carry evidence for low-frequency findings. Training it with frequency-weighted supervision should recover rare-finding accuracy at a fixed token budget better than retention based purely on visual change between slices.
This is a different use of frequency from the nearby work in the file. In BTB3D, "frequency" means spatial wavelet bands of voxel intensities, not how often words occur. CARVE allocates its budget by visual residuals, which measure how much a slice differs from its neighbours, and its own motivation is to avoid missing single-slice pathologies. It never asks which findings the language model could guess anyway. The obvious combination noted in the file is to put rare terminology in the prompt to guide pruning. That approach needs the finding to be known in advance, which it is not when writing a report. Our signal is computed at training time from reference reports and then distilled into a scorer that only looks at images.
Research Program
The frequency proxy is motivated by lexical simplification. There, raw corpus frequency is a baseline that context-aware models struggle to beat (SemEval-2012 Task 1), and rarity predicts human-rated complexity across ten language families (BEA 2024 MultiLS). We borrow the method from two further results. Biran et al. use cross-corpus frequency ratios. Sugihara et al. show that general and domain frequencies combined beat either alone for medical terms. Following them, we compare three proxies for how "prior-covered" a finding $y$ is. The first is raw label frequency in the training reports. The second is a ratio of that frequency to its frequency in reports of the same anatomical region, which separates "rare overall" from "rare for this organ." The third is the surprisal of $y$ under the blind model, $s(y) = -\log p_{\text{blind}}(y)$, where $p_{\text{blind}}$ is the probability the model assigns to the finding when given no visual tokens. We also test embedding norm. Liu et al. show that norm tracks token frequency and use it to set a curriculum in machine translation. Norm is cheap, but it has never been tested on medical findings.
The diagnostic study runs existing 3D vision-language models under a range of keep ratios. We test Hulu-Med-7B and Qwen3-VL 4B/8B, the backbones used by CARVE, and a BTB3D LLaMA-3.1 report generator from its public codebase. For each model we compute three quantities. The first is per-finding performance against training-report frequency. The second is the rank correlation between each finding's performance drop and its frequency. The third is a prior-agreement rate: how often the compressed model's predicted label set matches the blind model's predictions more closely than the full-token model's predictions do.
The method trains a lightweight scorer over visual tokens with backbone weights frozen. For each training volume we compute a target importance for each token $v$. It is the attribution of the reference report's finding tokens to $v$, weighted by the surprisal of each finding:
$$a(v) = \sum_{y \in R} s(y)\,\big|\nabla_{v} \log p(y \mid V)\big|,$$
where $R$ is the set of finding labels in the reference report and $V$ is the full set of visual tokens. The scorer learns to predict $a(v)$ from visual features alone. At inference it fills the token budget, either alone or merged with CARVE's slice-anchor tokens.
Baselines
The naive reference point is uniform depth subsampling at the same keep ratio, which is the cheapest way to fit a volume into a 2D-style backbone. Above it sits full-token inference, which is the upper bound we are trying to approach. The strongest existing competitor is CARVE. It is the only method in the file built for this exact problem, and its residual-retrieval branch is explicitly designed to keep single-slice pathologies. The reported numbers we must beat are its 98.1% of full-token performance across VQA benchmarks at a 20% keep ratio, and its 6.2-point margin over the strongest competitor on AMOS-MM report generation (both from CARVE). We will run CARVE and our scorer on the same backbones, the same splits and the same 20% and 10% keep ratios, with equal hyperparameter search. Because our scorer needs training and the file does not say whether CARVE does, we will report the extra training cost alongside the results. Beating CARVE means a statistically significant gain on the rarest third of findings while staying within one point of it on the most common third.
The key ablation trains the identical scorer with $s(y)$ fixed at 1, so every finding counts equally. This isolates the frequency weighting from the effect of simply having a learned scorer. We also swap the three frequency proxies and embedding norm against one another.
Why not just…?
Why not just keep more tokens or use a better tokenizer like BTB3D? That moves the operating point but does not show whether the bias goes away. We will run the frequency-stratified diagnostic at full tokens as well. If the rare-finding gap is already large there, compression is not the main cause and our method's claim narrows to the compressed regime. BTB3D reports a 40% relative clinical F1 gain over CT-CHAT and Merlin, but we found no reported absolute frequency-stratified result on Lacuna for BTB3D on CT-RATE. We will therefore measure it ourselves.
Why not just rely on CARVE's residual branch, which already targets small single-slice pathologies? This is the most serious threat. Visual residuals may already pick out the tokens that rare findings depend on. The ablation settles it. If CARVE's rare-tercile performance matches our scorer's, the method adds nothing. The diagnostic contribution would still stand, along with the evidence that residuals are enough.
Why not just subtract the blind model's predictions at decoding time, or reweight the fine-tuning loss by inverse frequency? Both are cheap ways to fight a language prior. Neither can restore evidence that pruning has already discarded: they adjust the model's confidence, not what the model sees. We will run both at matched budgets and also combine each with our scorer. If prior subtraction alone recovers rare findings, then the dropped tokens were not where the information lived. We would report that and reframe the work around decoding.
Proposed Experiments
Report generation uses CT-RATE, with Rad-ChestCT as external validation, and AMOS-MM. Both are scored by clinical F1 on disease labels, broken down by frequency tercile. VQA uses M3D-VQA and 3D-RAD. We add a counterfactual depth test in the style of the textbook-bias probes in Your other Left!. Using TotalSegmentator organ masks, we delete the slices that contain the organ a finding refers to. We then measure how often the model still reports the finding, which is a hallucination rate for that frequency band.
Three checks decide the outcome. First, if the drop under compression does not correlate with rarity, the premise fails and we stop. Second, the rare-tercile gain over CARVE must be significant at a matched budget. Third, the counterfactual hallucination rate must fall for rare findings without rising for common ones. Throughout, we keep in mind the lesson of AMOS, where a plain 3D-UNet (89.04% mDice) beat Swin-UNetr (86.32%). For that reason the simplest proxy, raw report frequency, is the default, and we adopt a more complex proxy only if it clearly wins.
References
- CARVE: Cross-Slice Anisotropic Reallocation of Visual Evidence for Efficient 3D Medical Volume Understanding
- Your other Left! Vision-Language Models Fail to Identify Relative Positions in Medical Images
- Better Tokens for Better 3D: Advancing Vision-Language Modeling in 3D Medical Imaging
- SemEval-2012 Task 1: English Lexical Simplification
- The BEA 2024 Shared Task on the Multilingual Lexical Simplification Pipeline
- Putting it Simply: a Context-Aware Approach to Lexical Simplification
- Semi-automatic Construction of a Word Complexity Lexicon for Japanese Medical Terminology
- Norm-Based Curriculum Learning for Neural Machine Translation
- AMOS: A Large-Scale Abdominal Multi-Organ Benchmark for Versatile Medical Image Segmentation