Who Pays the Interpretability Tax? Frequency-Gated Concept Bottlenecks for Long-Tailed Diagnosis

Interpretable medical and genomic models usually take one of two routes. Some force every prediction through human-readable intermediate quantities, or "concepts": clinical signs such as ulceration, gene-signature scores, or binding motifs. Others let an opaque network predict freely. The literature treats the accuracy cost of the first route as a single number, such as "within 1–2% of the black box." We think that number hides a split that depends on how common each class is. On common classes, a concept bottleneck (a layer the prediction must pass through, made of human-readable concepts) costs accuracy because it discards subtle cues the concepts do not name. On rare classes, the same bottleneck can improve accuracy, because a small set of named concepts is a strong prior when examples are scarce. Rare classes are also where clinicians most need to audit and correct a prediction.

We propose to measure this split directly, as a curve of the gap between the black box and the concept model plotted against how many training examples a class has. We then use the curve to build a model with a per-class opaque side channel. Frequent classes may draw on the side channel. Rare classes must be predicted from concepts alone. The practical claim is that interpretability does not need to be bought uniformly. It can be concentrated on the tail of the class distribution, where it both costs less and helps more.

Core Claim

The file already contains evidence pointing in both directions, and that disagreement motivates the work. On the side of bottlenecks helping, Koh et al. introduced concept bottleneck models, which predict concepts from the input and then the label from those concepts. They report that a bottleneck model trained on 25% of the osteoarthritis data matched a standard model trained on 100%. HybridCBM combines a fixed bank of language-model concepts with learned ones and beats unconstrained CLIP linear probes at 1–4 examples per class. In genomics, SAM restricts splicing prediction to a sparse set of RNA-binding-protein motifs. It trails SpliceAI overall (79.0% vs ~93% top-k accuracy) but outperforms it on rare, evolutionarily new exons.

On the side of bottlenecks hurting, Veltmeijer et al. trained a melanoma classifier to also predict asymmetry, border and color features. They found this auxiliary concept prediction degraded accuracy on benign nevi, the majority class. Koh et al. also report a joint bottleneck error of 19.9% versus 17.5% end-to-end on the bird-species benchmark CUB.

Our reading is that these results are compatible. The cost of a bottleneck falls on data-rich classes and the benefit goes to data-poor ones. No paper in the file measures the tradeoff as a function of class frequency, and none uses that function to decide where opacity is allowed. The claim is falsifiable. If the black box trained with a class-balanced loss beats the concept model on the tail, the crossover does not exist and the gating idea is unnecessary.

Research Program

The first component is a measurement study. We take benchmarks that have concept labels and naturally long-tailed diagnostic labels. For each, we also run controlled sweeps in which one diagnostic class is subsampled to $n \in \{5, 10, 25, 50, 100, \text{full}\}$ training examples while everything else is held fixed. At each point we compute the per-class gap in recall and in area under the precision-recall curve (AUPRC) between a black box and a concept model. We then locate the crossover count $n^\ast$ below which the concept model wins.

We use AUPRC and per-class calibration rather than overall accuracy or AUROC. The papers on evaluation under class imbalance and on AUROC versus AUPRC argue that aggregate discrimination metrics mislead on skewed medical data. That is exactly the kind of averaging that could hide the split we are looking for.

The second component is the model itself. Let $\hat c(x)$ be the predicted concept vector. Let $r(x)$ be a low-dimensional residual embedding from the same backbone, with gradients blocked from flowing into the concept head. The logit for class $k$ is

$$z_k(x) = w_k^\top \hat c(x) + g_k\, u_k^\top r(x),$$

where $w_k$ and $u_k$ are class-specific weights and $g_k \ge 0$ is a learned per-class gate on the residual path. Training minimizes the diagnostic loss plus concept losses, plus a penalty $\lambda \sum_k g_k (N/n_k)^\beta$. Here $n_k$ is the number of training examples of class $k$, $N$ is the total, and $\beta \ge 0$ controls how steeply the penalty grows as a class becomes rarer. We also test a hard variant that sets $g_k = 0$ whenever $n_k < n^\ast$, using the crossover estimated in the measurement study.

Two details keep the concept path honest. First, the concept head is trained with per-concept positive weighting. SkinCon shows that 26 of its 48 concepts occur in fewer than 50 images, and rare classes tend to depend on exactly these rare concepts. Second, the concept path is fed hard or independently trained concept predictions, following the independent and sequential variants in Koh et al.. This prevents task information from leaking through concept scores that are supposed to mean something specific.

The result is a model whose explanation for any tail-class prediction is a sparse linear readout of named concepts, which a clinician can correct at test time. Head classes keep the accuracy that a pure bottleneck gives up.

The genomic arm applies the same test on the SCOPE breast cancer single-cell atlas: 138 patients and 274,555 cells. SCOPE maps TranscriptFormer embeddings through a patient-level aggregator to concepts aligned with curated gene signatures and immune pathways. Its concepts are computable for every patient, so there is no annotation bottleneck. We add the gated residual to SCOPE's concept projector and ask whether rare subtypes or grades behave like the rare image classes. SCOPE did not study this question. Its model was trained for one epoch and its downstream tasks were evaluated without attention to imbalance. With only 138 patients, per-class estimates will be noisy. We therefore treat this arm as a transfer test with patient-level bootstrap intervals, not as the primary evidence.

Baselines

The naive reference is a black-box backbone trained with plain cross-entropy, with no rebalancing and no concepts. It shows how badly the tail collapses, the majority-class failure documented across medical imbalance work.

The strongest competitor on accuracy is the same backbone trained with a class-balanced loss, either focal or inverse-frequency weighted. It is the most likely model to erase any tail advantage the concepts appear to have. We found no reported result on Lacuna for a focal-loss black box on SkinCon, Derm7pt or LIDC-IDRI, so we will run it ourselves.

For reference points on the benchmarks themselves:

  • On LIDC-IDRI, PN-SAMP, a multi-task 3D network predicting nodule attributes and malignancy together, reports 97.58% malignancy accuracy.
  • On SkinCon's DDI subset, the post-hoc bottleneck reaches AUC 0.639 against 0.595 for the black-box DeepDerm.
  • CCBM, which adds learned "unknown concept" adapters, reports 93.96% concept-detection AUC on Derm7pt and a classification gap to black boxes of 1–2%.
  • On the single-cell atlas, SCOPE reports accuracy 0.68 and F1 0.67 for subtype classification.

The strongest interpretable competitor is CCBM, because its unknown-concept adapters are a global residual channel. It is the natural foil to a class-specific one. We will also run a plain Koh-style bottleneck model with per-concept positive weighting.

The key ablations isolate the gate. The first sets a single shared $g$ for all classes, which yields a CCBM-like hybrid. The second removes the frequency penalty ($\beta = 0$). The third is a hard routing ensemble that uses the black box when its predicted class is frequent and the concept model otherwise.

For a fair comparison, every model shares one backbone and pretrained initialization, one set of data splits, and an equal hyperparameter-search budget. Every model receives the same per-concept and per-class reweighting wherever the architecture has a place for it, so that rebalancing is not confounded with architecture.

Beating the baselines means two things. On head classes, the gated model must stay within one point of the class-balanced black box in recall and AUPRC. On classes below $n^\ast$, it must match or exceed the plain bottleneck model's recall. It must also retain the plain model's test-time intervention gain, and its tail predictions must show no measurable residual contribution.

Why not just…?

Why not just train a black box with a class-balanced loss and explain it with attribution maps afterward? This is the most serious threat, and the measurement study is designed to settle it. If the rebalanced black box matches the concept model on the tail across the subsampling sweep, our crossover claim fails. Even if it wins, two things remain true. Attribution maps cannot be intervened on the way concepts can: correcting two concepts cut osteoarthritis error from over 0.4 to about 0.3 in Koh et al.. Black-box features can also latch onto confounders: raw BioViL features scored 9.4% against 50.7% for concept-grounded predictions on the confounded NIH-age benchmark in Yuksekgonul et al.. We will report both outcomes rather than assume one.

Why not just use a hybrid bottleneck with a global residual, as CCBM or HybridCBM do? A shared residual is available to every class. The loss is dominated by head classes, so the residual will likely be shaped by them, and rare-class predictions can then quietly route through it. We will measure this directly by reading out how much of each tail-class logit comes from the residual in CCBM versus our gated model. If CCBM's residual turns out to be naturally inert on the tail, the per-class gate adds nothing, and we will say so.

Why not just rebalance the concepts with per-concept loss weights? This is cheap and probably necessary, so every concept model in the study gets it. It targets a different failure, though: rare concepts that the model fails to detect. It does not address head classes losing accuracy to the bottleneck, which is the distraction effect in Veltmeijer et al.. The ablation that combines concept reweighting with no gate shows how much of any gain comes from reweighting alone.

Why not just balance the classes with generative augmentation? Synthesizing rare cases requires reproducing plausible combinations of many co-occurring concepts. The file's evidence suggests generators memorize or collapse modes in this setting. We include GAN-augmented training of the plain bottleneck model as a secondary baseline, trained on Derm7pt and ISIC 2017, where Veltmeijer et al. found augmentation useful for melanoma.

Proposed Experiments and Decisive Checks

The image experiments use four datasets:

  • SkinCon, which has 48 concepts and long-tailed diagnoses across Fitzpatrick 17k and DDI; we report subgroups by skin tone as well.
  • Derm7pt, with its seven dermoscopic concepts.
  • LIDC-IDRI, with eight or nine nodule attributes and malignancy scores; the rare malignancy grades form the tail.
  • CUB-200-2011, as a non-medical control where the bottleneck's head-class cost is already known.

The genomic experiment uses the SCOPE breast atlas. Its concepts are signature scores, and subtype and grade provide the labels. If time allows, a retrospective reanalysis of SAM's canonical versus new-exon split can serve as a third, sequence-level instance of the same curve.

For every model we report per-class recall, AUPRC and expected calibration error, each plotted against $\log n_k$. We also report the gain in task accuracy after correcting the $m$ most uncertain concepts, stratified into head and tail classes. Finally, we report residual attribution per class.

The first decisive check is whether the gap curve crosses zero. The concept model should beat the rebalanced black box below some $n^\ast$ on at least three of the four image datasets, with bootstrap intervals that exclude zero. The second check is whether the gated model sits on the upper envelope of both curves, and whether the learned $g_k$ tracks $n_k$ without being told to under $\beta = 0$. A gate that tracks frequency on its own would be strong evidence that the split is real rather than something our penalty imposes. The third check is whether tail-class intervention efficacy survives. If adding the gate reduces the intervention gain on rare classes, the side channel is leaking and the interpretability guarantee is void.

If the first check fails, the answer to the research question is that a rebalanced black box dominates at every class frequency on these benchmarks. That negative result would still be useful. It would take away the sample-efficiency argument for concept bottlenecks in medicine and leave auditability as their only justification.