Describe Like a Radiologist, Decide From Pathology: Radiomics-Grounded Concept Bottlenecks for Lung Nodules

Most "explainable" medical imaging models explain themselves after the fact. A deep network makes a prediction, and a separate tool such as a heatmap or a feature-importance score tries to say why. Systematic tests show these after-the-fact explanations often do not reflect what the model actually computed. We propose a lung-nodule malignancy model whose reasoning chain can be read end to end. It starts from radiomic features, which are measurements computed from the CT scan by fixed formulas: shape, intensity statistics, and texture. A small, sparse model maps these measurements onto the descriptive words radiologists already use, such as spiculation, lobulation, margin sharpness, internal texture, calcification and sphericity. A second sparse model maps those words onto malignancy.

The training signal is split on purpose. The descriptive layer learns from radiologists' attribute ratings, since describing what a nodule looks like is something radiologists do reliably. The malignancy layer learns only from biopsy- or outcome-confirmed labels, since radiologists' malignancy guesses on public data are known to be heavily biased toward calling nodules cancer. The result is a model a clinician can inspect at every step. The clinician can see which measurements make the model call a nodule "spiculated," and how much spiculation pushes the risk up. If the clinician disagrees with a description, they can override it and watch the risk update. The open question is whether this costs meaningful accuracy against the best black-box nodule models. This proposal is built to answer that question.

Core Claim

The obvious combination at this intersection is "radiomic features as the concept bottleneck." A concept bottleneck model, introduced in Concept Bottleneck Models, first predicts human-readable concepts from the image and then predicts the outcome only from those concepts. Using radiomic features as the concepts sounds new, and the research file's gap analysis notes that no paper in it does this. On reflection, though, it is nearly empty. Radiomic features are computed exactly from the image, so there is nothing for a bottleneck to learn on the way to them. A sparse regression on radiomic features is already the "bottleneck model," and that is what every radiomics nomogram is. Its concepts, such as "GLCM joint entropy at wavelet band LHL," also cannot be meaningfully corrected by a radiologist at the bedside.

Our claim is different. Radiomics should sit below the concept layer as its transparent input, not be the concept layer. This changes two things relative to standard concept bottlenecks.

The first change is that the step from image to concept is itself auditable. Each clinical concept is a short, readable function of named measurements, rather than the output of an opaque network head. That opaque head is exactly the part of a standard concept bottleneck nobody can inspect.

The second change is that concept supervision and outcome supervision come from different sources. Concepts are learned once on a cohort with dense radiologist attribute ratings (LIDC-IDRI). They are then applied without new annotation to cohorts that have only pathology or outcome labels.

We predict three results. The model will come within about 0.02 area under the ROC curve (AUC) of strong black-box models on held-out and external cohorts. Its explanations will be far more stable than SHAP applied to tree models over raw radiomics. And radiologist interventions on its concepts will measurably improve predictions.

Where This Sits in Prior Work

Three lines of work motivate the design.

Radiomics pipelines show that formula-based features carry real signal. Highly accurate model for prediction of lung nodule malignancy with CT scans reports a mean AUC of 0.91 on LIDC-IDRI from hand-crafted quantitative features alone, rising to 0.993 when they are concatenated with CNN features. Those labels are radiologist ratings, though, and the concatenated model has no interpretable structure. Interpretable-radiomics studies such as the axillary lymph node MRI model (test AUC 0.859) and the pancreatic recurrence model (external AUC 0.884) both use random forests explained afterwards with SHAP. We keep their features but drop the after-the-fact explainer.

The second line shows that after-the-fact explanations fail. Guidelines and evaluation of clinical explainable AI in medical image analysis tested 16 heatmap methods on brain and knee MRI and found that none reliably met truthfulness or informative plausibility. The BI-RADS 4 reader study found that radiologists treated heatmaps as sanity checks, while numeric certainty scores drove their revisions. The file also notes a known weakness of perturbation explainers on correlated features: they evaluate impossible feature combinations. Radiomic features are notoriously collinear, so this applies directly. These results constrain the design. Our explanations must be exact by construction, not estimated.

The third line supplies the architecture and its known failure points. Concept Bottleneck Models showed on knee radiographs that correcting two concepts cut task error from over 0.4 to about 0.3. Concept Complement Bottleneck Model closed the gap to black boxes to within 1–2% by adding an unsupervised "unknown concept" branch. It reported instability on LIDC-IDRI and small datasets, and its discovered concepts have no names. Hybrid Concept Bottleneck Models similarly mixes fixed and learnable concepts. Our design forbids unnamed concepts in the main model. It handles incompleteness by measuring it explicitly with a separate residual audit, rather than absorbing it silently.

For the fitting procedure we borrow the reluctant additive strategy of RAITR. That method fits a sparse linear model first and adds smooth nonlinear terms only when a concordance-based criterion supports them. RAITR was built for treatment rules on genomic data. We apply it to both layers of an imaging model.

The supervision split rests on Re-thinking and Re-labeling LIDC-IDRI. That paper showed that a model trained on LIDC radiologist malignancy scores reaches sensitivity 0.97 but specificity only 0.23 against biopsy truth. It also re-classified more than half of score-4 nodules as benign. Parse and Recall (PARE) is the strongest accuracy reference. Its "prototypes" resemble radiologist recall but are latent clusters with no radiomic or clinical definition.

Research Program

The model has two stages.

Stage 1: from measurements to concepts. For each nodule $i$, we compute a standardized radiomic vector $r_i$ from a 3D segmentation. It covers first-order, shape, co-occurrence, run-length, wavelet and Laplacian-of-Gaussian features. We then reduce it to correlation clusters, so that each retained feature stands for a family of near-duplicates. This step directly addresses the problem of having far more features than patients.

For each LIDC-IDRI semantic attribute $k$, we fit a reluctant additive predictor $\hat c_{ik} = h_k(r_i)$. Each $h_k$ is restricted to a small number of named terms, so the concept's "definition" can be printed on one line. Concept targets are the mean ratings across the up to four LIDC readers. Their disagreement gives a natural ceiling: no concept head should be expected to exceed inter-reader agreement.

Stage 2: from concepts to malignancy. A second reluctant additive model maps $\hat c_i$ to malignancy. It is trained only on nodules with confirmed labels, drawing on three sources: biopsy-proven SCH-LND nodules, NLST nodules with screening outcomes, and LIDC nodules whose labels were propagated from biopsy anchors following the re-labeling paper.

Because the model is additive in a handful of named concepts, each concept's contribution to a patient's risk is exact. A clinician override replaces $\hat c_{ik}$ with their own reading and recomputes the risk. No sampling-based explainer is involved.

Residual audit. Separately, we train a "residual" variant. It adds a CNN embedding passed through a narrow linear channel, and we report the share of predictive gain that flows through that channel. This answers the incompleteness question honestly. If the named concepts miss malignancy-relevant signal, the residual will carry it and we will report how much.

Robustness checks. Radiomics is known to be sensitive to scanner settings and segmentation. We therefore quantify the shift between cohorts with Fréchet radiomic distance and test whether concept-head degradation tracks that distance. We also perturb segmentations by dilation, erosion and alternate reader contours to measure how much the concepts move.

Baselines

Naive reference points. The first is nodule diameter plus age in a logistic model. The second is a model trained on LIDC radiologist malignancy scores and evaluated against biopsy truth. The file reports the latter at sensitivity 0.97 and specificity 0.23 in the re-labeling paper, whose re-labeled model reaches 0.67 and 0.66. Our model must beat the re-labeled operating point on SCH-LND-anchored evaluation to show that the supervision split helps.

Strongest accuracy competitor: PARE. PARE reports test AUC 0.928 on NLST, 0.931 on its in-house noncontrast CT cohort, and 0.801 on the external LUNGx challenge. We cannot obtain its in-house data. We therefore re-run PARE, or the closest reproducible version if code is unavailable, on our exact NLST splits and on LUNGx. It gets the same segmentations or context masks and the same hyperparameter search budget, measured in GPU-hours, as our model's CNN-based ablations. We will report published and re-run figures side by side. "Beating" PARE is not the goal. The goal is an NLST AUC within 0.02 of the re-run PARE, and a LUNGx AUC at or above it, since a sparse feature model may transfer better than a large network.

Strongest radiomics competitor. This is a random forest over the full radiomic vector, explained with TreeSHAP. It matches the axillary node and pancreatic recurrence studies. It also stands in for the hybrid CNN-plus-radiomics recipe of NoduleX, whose standalone radiomics AUC was 0.91 on LIDC radiologist labels. Both versions are tuned with the same budget and the same confirmed labels.

Strongest interpretable-architecture competitor. This is a standard concept bottleneck whose concept head is a 3D CNN trained on the same LIDC attributes, plus the Concept Complement Bottleneck Model. The file reports only that CCBM is "within 1–2% of black-box models" on LIDC; it gives no individual AUC for that dataset.

Ablations. Three ablations isolate the key ingredients. The first removes the concept layer, giving a reluctant additive model directly on radiomics. The second keeps the concept layer but trains stage 2 on radiologist malignancy scores instead of confirmed labels. The third replaces reluctant fitting with a plain lasso, which tests whether the nonlinear terms earn their place.

Why not just…?

Why not just fit a lasso nomogram on radiomic features and call it interpretable? This is the most serious threat. It may match our AUC, and our first ablation is exactly this model. If it does, accuracy alone cannot separate the two. The difference shows up in two experiments the nomogram cannot pass. The first is intervention. A radiologist cannot sensibly correct a wavelet entropy value, but can correct "spiculated: no," so we measure the gain from correcting concepts on held-out LIDC nodules with propagated biopsy labels. The second is a reader study modeled on the BI-RADS 4 design, which tests whether concept-level explanations change correct revisions more than coefficient lists do. If neither shows a difference, we will conclude the semantic layer is cosmetic.

Why not just train a standard concept bottleneck from pixels on LIDC attributes? That model's concept head is a black box, so "spiculation 0.8" is itself unexplained. Its soft concept scores can also leak non-concept information, a known failure noted in the file. CCBM's reported instability on LIDC-IDRI suggests pixel concept heads struggle in this regime. We compare concept AUCs head to head. If CNN concept heads are much more accurate, that is a real cost we will report, along with the residual audit showing how much of the gap is unnamed signal.

Why not just use a strong deep model with SHAP or heatmaps? The clinical XAI guidelines evaluation found that no tested heatmap method met truthfulness. We additionally measure explanation stability across bootstrap retrainings, using rank correlation of per-patient attributions, for TreeSHAP over correlated radiomics versus our exact additive contributions. If after-the-fact explanations prove stable here, the case for our design narrows to intervention and auditability alone.

Decisive Checks

We will consider the idea vindicated if all four of the following hold. First, NLST AUC is within 0.02 of the re-run PARE, and LUNGx AUC is no worse than PARE's. Second, on LIDC, concept-head AUC per attribute approaches the inter-reader ceiling. Third, correcting the two most-erroneous concepts per nodule improves biopsy-anchored AUC significantly, mirroring the effect in Concept Bottleneck Models. Fourth, the residual channel carries less than a quarter of the gain over the naive baseline.

The idea fails in two cases. One is if the residual channel carries most of the signal, which would mean radiologist vocabulary does not cover what predicts malignancy. That finding is itself worth publishing. The other is if the nomogram ablation matches the full model on accuracy, intervention and reader outcomes alike.