Radiologists judge a lung nodule using evidence at three spatial scales. The first is the nodule's own shape and margin, at millimetre detail. The second is its neighbourhood: how close it sits to vessels and the pleura (the membrane lining the lung), and whether airways run into it. The third is the whole scan, such as which lobe the nodule is in and how many other nodules are present. Deep networks for 3D CT cannot hold a full high-resolution scan in GPU memory. They therefore either crop a small patch around the nodule, which discards the outer two scales, or shrink the whole scan, which erases the thin vessels and fine margins that the inner two scales depend on. Interpretable models then make this worse. A concept bottleneck is a network that first predicts human-readable attributes and then predicts the diagnosis from those attributes alone. Current concept bottlenecks predict attributes as learned scores from a patch, so a clinician cannot tell whether a "close to a vessel" concept reflects a real vessel or a texture shortcut.
We propose a diagnostic model with two properties. Its relational concepts are physical measurements: distances, branch counts and lobe membership, computed by differentiable geometry from anatomy segmentations. Each concept is also computed at the resolution its evidence needs: full resolution in a thin shell around the nodule, coarse resolution for the whole scan. The central empirical bet ties in class imbalance. The error in these concepts comes mostly from the segmentation of thin structures, because vessels and small airways occupy a tiny fraction of voxels. Under-segmenting them pushes measured distances upward, which makes malignant nodules look benign. An interpretable model is therefore only as trustworthy as its imbalance handling on the anatomy it measures. We test that claim directly against reference measurements.
Core Claim
A three-scale concept bottleneck can match a black-box context model on lung nodule malignancy prediction while producing concept values that can be checked against ground truth. The scales are high-resolution nodule morphology, high-resolution peri-nodular geometry, and coarse whole-scan context. Getting there requires two things that existing work treats separately. The first is spending full resolution only where a concept's evidence lives, which is the answer to the memory constraint. The second is training the thin-structure segmenter with an imbalance-aware objective and a concept-consistency term, which is the answer to class imbalance, because measurement concepts inherit segmentation bias. The claim is falsifiable at the concept level as well as the diagnosis level. We can measure in millimetres how far each concept is from a manually delineated reference.
Positioning
Three papers motivate the gap. Joint Learning for Pulmonary Nodule Segmentation, Attributes and Malignancy Prediction (PN-SAMP) predicts nine radiologist attributes and malignancy from a cropped 3D nodule patch. It shows that a 3D attribute bottleneck can be accurate, but it never sees the nodule's surroundings. It was also evaluated only on 1,404 consensus nodules, with ambiguous cases removed. Relationship between pulmonary nodule malignancy and surrounding pleurae, airways and vessels measures distances and branch counts offline on LIDC-IDRI. It finds that malignant nodules sit closer to vessels than benign ones (1.07 mm versus 1.42 mm) and have more airway branches nearby. This is the strongest evidence that relational measurements carry signal. However, it is a disjoint pipeline on pre-cropped regions, and its small distance gap is exactly what segmentation bias could erase. Parse and Recall (PARE) segments surrounding vessels, parenchyma and trachea and fuses them with nodule features through attention. It reaches an area under the curve (AUC) of 0.928 on NLST and 0.801 on external LUNGx. Its context is consumed by attention rather than exposed as measurements, and it is limited to the immediate neighbourhood.
Two papers supply methods we reuse. The Balance Loss of A Unified Loss for Handling Inter-Class and Intra-Class Imbalance pairs foreground voxels with an equal number of the hardest background voxels and down-weights easy interior voxels. Its analysis shows that tiny soft background probabilities accumulate into a bias that thresholded predictions do not have. We apply it to vessels and airways, and we argue below that the same soft-versus-hard bias reappears inside soft geometric concepts. No More Sliding Window (NMSW) shows that a coarse global pass followed by a few high-resolution patches can match sliding-window inference at a fraction of the compute. Our allocation is simpler and more interpretable than its learned sampling: high resolution goes to the concept shell, a region defined by what is being measured.
Two papers constrain the design. Concept Complement Bottleneck Model (CCBM) reports that regression-style concepts on LIDC-IDRI are unstable under intervention, and that rare concepts get swamped by easy ones. This is part of why we replace learned scores with measurements wherever a measurement exists. Lesion-Centric Latent Phenotypes derives morphology from predicted masks in 2D ultrasound, which implies that mask boundary errors propagate into its phenotypes. We test that propagation explicitly in 3D.
Research Program
The model has three branches sharing one low-resolution pass over the whole thoracic CT. The coarse branch segments lungs, lobes and candidate nodules at reduced resolution. From these it computes scan-level concepts: lobe of the nodule, number of other nodules, and fraction of low-attenuation lung as an emphysema proxy. The morphology branch processes a full-resolution nodule patch and predicts the radiologist-rated LIDC attributes (spiculation, lobulation, margin, sphericity and others), as PN-SAMP does. Rare high ratings in this branch are handled with a class-balanced concept loss. The relational branch processes a full-resolution shell extending a fixed number of millimetres beyond the nodule surface. It segments vessels, airways and pleura there with Balance Loss, and computes distances and branch counts. If the nearest pleura lies outside the shell, the coarse branch supplies its distance, since coarse resolution is adequate at large distances. A sparse logistic layer maps all concepts to malignancy. An optional small residual channel is reported separately, so that any accuracy it adds is visible as non-interpretable.
The relational concepts must be differentiable so that diagnostic and concept errors can train the segmenter. The distance from the nodule surface $S$ to a structure with predicted voxel probabilities $p_v$ at positions $x_v$ is computed as a soft minimum:
$$d_\tau = -\tau \log \sum_{s \in S}\sum_{v} p_v \, e^{-\lVert x_v - s\rVert/\tau},$$
where $\tau$ is a temperature in millimetres. This formula makes the imbalance link concrete. The shell holds many background voxels with small but nonzero $p_v$, and they pull $d_\tau$ toward zero. This is the soft-versus-hard bias identified by the Balance Loss paper, now acting on a concept rather than a Dice score. Conversely, thresholded thin vessels that the segmenter missed push the hard distance upward. We therefore add a consistency term that penalises the gap between $d_\tau$ and the distance computed from the thresholded mask, and we report both quantities.
Baselines
The naive reference is a 3D convolutional network trained on a fixed nodule crop with plain cross-entropy on malignancy. A second naive reference downsamples the whole scan to fit memory. Together they bracket "context ignored" and "detail ignored."
The strongest existing interpretable competitor is PN-SAMP, reimplemented on the same patient-level LIDC-IDRI splits. Its reported 97.58% malignancy accuracy is on consensus-filtered nodules. We evaluate it twice, on that filtered subset and on the full set including ambiguous nodules, and we compare against it only within matching subsets.
The strongest context-using competitor is PARE. Its reported AUC is 0.928 on NLST and 0.801 on external LUNGx. We retrain a PARE-style model on our splits with the same backbone family, an equal GPU-hour budget and an equal hyperparameter search budget, and we compare on LUNGx with the same external protocol. Beating it means matching its AUC within the confidence interval while exposing every relational input as a measurement. Exceeding it on the small-nodule and atypical-malignancy subgroups would be a bonus, not the core claim.
The offline pipeline of the pleura/airway/vessel study is automated with off-the-shelf segmentation and logistic regression. It is the most important comparison, because it shares our concepts but not our training.
For imbalance, the Balance Loss paper reports 66.8 Dice against 59.8 for cross-entropy on KiTS19 kidney tumours. No reported result was found on Lacuna for Balance Loss on pulmonary vessel or airway segmentation, so that comparison is run here.
Ablations isolate the key ingredients: Balance Loss swapped for Dice or cross-entropy in the relational branch; the consistency term removed; each scale removed in turn; and the relational shell run at half and quarter resolution.
Why not just…?
Why not just automate the offline measurement pipeline and fit logistic regression? This is a real threat. If a strong vessel segmenter followed by hand-computed distances matches our end-to-end model, then end-to-end training adds nothing. The contribution would shrink to the scale-matched concept set and the faithfulness audit. The ablation that trains our segmenter without diagnostic or consistency gradients is exactly this pipeline, so the experiment settles the question. We will report the outcome either way. The audit is what tells a clinician whether any version of these concepts can be trusted.
Why not just crop a larger patch and use a black-box network with Grad-CAM? A larger crop does capture neighbourhood context at modest memory cost. That is why our shell is local and not scan-wide. What it does not provide is a verifiable statement such as "2.1 mm to the nearest vessel." A heatmap over the shell cannot be checked against a reference distance. The latent-phenotype work reports that unconstrained saliency drifts onto background. We include a large-crop black-box model under equal compute. If it is clearly more accurate, the interpretability cost is quantified rather than hidden.
Why not just add class-weighted losses to PN-SAMP's attribute heads? That treats imbalance in the concept labels, which we also do in the morphology branch. It cannot fix relational concepts, because no radiologist label exists for "distance to vessel." The bias there enters through the thin-structure segmentation. The resolution and loss sweeps in the audit show whether that segmentation bias is large enough to matter.
Proposed Experiments
The study uses LIDC-IDRI with patient-level splits, the radiologist malignancy ratings, and the eight to nine semantic attributes. LUNGx serves as the external test, following PARE. For the faithfulness audit we manually delineate vessels, airways and pleura at full resolution inside the peri-nodular shell for roughly one hundred LIDC nodules, stratified to over-represent malignant and small nodules. This annotation is our main cost.
The first decisive check is the audit itself. We measure millimetre error of each relational concept against reference measurements as a function of shell resolution and segmentation loss. We also test whether the malignant-versus-benign vessel distance gap reported offline survives automated measurement. Our hypothesis predicts two things: downsampling and plain cross-entropy inflate distances for malignant nodules specifically, and Balance Loss with the consistency term recovers the gap.
The second check is diagnostic AUC and calibration on the full LIDC set, on LUNGx, and on two subgroups where imbalance bites: nodules under 6 mm, and malignant nodules with low spiculation ratings.
The third check is concept intervention. We replace predicted relational concepts with the reference measurements and record the change in diagnostic AUC. If intervention helps substantially, the bottleneck is genuinely using the concepts and the remaining work is segmentation. If it does nothing, the concepts are decorative and the claim fails.
References
- Joint learning for pulmonary nodule segmentation, attributes and malignancy prediction
- Relationship between pulmonary nodule malignancy and surrounding pleurae, airways and vessels: a quantitative study using the public LIDC-IDRI dataset
- Parse and Recall: Towards Accurate Lung Nodule Malignancy Prediction Like Radiologists
- A Unified Loss for Handling Inter-Class and Intra-Class Imbalance in Medical Image Segmentation
- No More Sliding Window: Efficient 3D Medical Image Segmentation with Differentiable Top-K Patch Sampling
- Concept Complement Bottleneck Model for Interpretable Medical Image Diagnosis
- Lesion-Centric Latent Phenotypes from Segmentation Encoders for Breast Ultrasound Interpretability