Concept bottleneck models are image classifiers built to be readable. They first predict human-named attributes ("red wing", "hooked beak") and then predict the label from those attributes alone. This lets a person inspect a decision and correct a wrong attribute at test time. In practice, the most accurate versions use continuous attribute scores, and those scores quietly carry extra label information that the attribute names do not describe. This is called leakage. When a human corrects an attribute, that hidden information is erased, and accuracy can fall rather than rise. The usual fix is to force attributes to be hard 0/1 values. That restores trustworthy interventions but costs accuracy whenever the named attributes are incomplete.
This proposal asks a question nobody has tested: how much of that hidden channel is memorization of specific training images, and how much is a real, generalizable signal the attributes simply failed to name? We answer it with a tool from generalization theory called loss-difference conditional mutual information. The idea is simple. Pair each training image with an unseen "ghost" image, then measure how well the gap between their losses reveals which of the two the model was trained on. We split that quantity into what the hard attributes carry and what the leaked residual adds on top. If leakage is mostly memorization, we can penalize only the memorized part of the residual. That would keep the generalizable extra signal and the accuracy it buys, while restoring interventions that work.
Core Claim
Margeloiu et al. diagnose leakage in Addressing Leakage in Concept Bottleneck Models. They fix it with hard autoregressive concept predictors and an explicit side-channel vector, and they measure an empirical completeness score. They never ask whether the leaked information overfits. Generalization theory supplies the missing measurement. Wang and Mao, in Tighter Information-Theoretic Generalization Bounds from Supersamples, show that the mutual information between a train-or-ghost selection bit and a single scalar loss difference bounds the generalization gap tightly. In the zero-training-error regime this bound decays at a fast rate. Our claim has two parts. First, the chain rule for mutual information separates a concept model's memorization into a hard-concept part and a leakage part. Second, the leakage part predicts how badly human interventions will fail.
The decomposition is the following. Let $U$ be the hidden bit indicating which member of each supersample pair was used for training. Let $\Delta L$ be the train-minus-ghost loss difference of the full model (soft concepts plus any side channel). Let $\Delta L_h$ be the same difference when concepts are thresholded to 0/1 and the side channel is zeroed. Then:
$$I(U;\Delta L,\Delta L_h) \;=\; I(U;\Delta L_h) \;+\; I(U;\Delta L \mid \Delta L_h).$$
The first term is memorization that flows through the human-readable concepts. The second term, which we call leakage memorization, is the sample-specific information the residual adds once the hard pathway is known. With 0-1 losses each difference takes three values, so both terms are plug-in estimates over a nine-cell table. This avoids the weight-space intractability that Information-theoretic generalization bounds for black-box learning algorithms was designed to escape.
The prediction is falsifiable. Across models that differ in joint-loss weight and side-channel width, leakage memorization should rank intervention damage better than the raw soft-versus-hard accuracy gap does. Penalizing it should then recover soft-model accuracy with hard-model intervention behaviour. If instead the conditional term stays small while intervention damage is large, leakage is a generalizable shortcut rather than overfitting. That would also be a clean, publishable answer, and it would rule out regularization as a fix.
Research Program
The first stage is measurement. We train concept models on supersamples, following the construction of Tighter Information-Theoretic Generalization Bounds from Supersamples. Each draws $n$ pairs and trains on one member of each pair, chosen by random $U$, repeated over $k$ draws of $U$. We vary joint versus sequential versus independent training as in Concept Bottleneck Models, sweep the joint weight $\lambda$, and vary side-channel width as in Margeloiu et al.
For each model we estimate both terms of the decomposition, the total, and the class-wise version of the leakage term following Class-wise Generalization Error: an Information-Theoretic analysis. That paper found generalization error varying fourfold across classes on balanced CIFAR-10. We expect leakage to concentrate in classes whose distinguishing features the attribute vocabulary misses.
The second stage is a training penalty. Lyu et al., in Recognizable Information Bottleneck, train a critic to tell training representations from ghost representations and penalize the encoder when the critic succeeds. Their penalty acts on the whole latent vector, which is not human-interpretable and is not tied to leakage. We apply the same recognizability critic only to the residual: the soft-minus-hard concept difference plus the side-channel vector. The hard concepts stay unpenalized and supervised by attribute labels. The residual may carry whatever generalizes but is pushed to carry nothing that identifies individual training images.
Ghost data costs labeled examples. Every baseline below therefore receives the ghost set as additional training data, so the penalty gains no data advantage.
Baselines
The naive reference points are a standard end-to-end network and a joint soft concept model without any fix. For these, Koh et al. report 17.5% and 19.9% task error respectively on CUB-200-2011 in Concept Bottleneck Models. We retrain both with our backbone and data split rather than copying numbers, and we report the published figures only as a sanity check.
The strongest existing competitor is the Hard AR model with side channel from Addressing Leakage in Concept Bottleneck Models. It is the only method in the file that directly targets the accuracy-versus-intervention tension. The paper reports 81.4% concept accuracy on CUB, against 48.1% for a soft independent model, with label accuracy within about 1% of soft models. We run it from its published design with the same backbone, the same 312-attribute CUB annotation, the ghost data folded into training, and the same hyperparameter search budget as our method.
Beating it means two things together. Our task accuracy must exceed Hard AR's by more than seed variance. Our intervention curve, task accuracy as an increasing number of concepts are set to ground truth, must lie on or above Hard AR's at every intervention budget.
The RIB penalty applied to the entire concept-plus-side-channel vector is a second competitor and doubles as the key ablation. If whole-vector penalization matches residual-only penalization, the decomposition adds nothing to the method. Further ablations isolate the remaining ingredients. A width-limited side channel with weight decay and no critic tests whether plain capacity control suffices. A critic trained against shuffled $U$ tests whether the gain comes from the train/ghost signal or from generic noise injection.
For the diagnostic question, the baselines are the soft-versus-hardened accuracy drop on a validation set, the plain train-test gap, and Margeloiu's completeness score.
Why not just…?
Why not just measure the accuracy drop when concepts are hardened on a validation set? This is the real threat to the diagnostic half of the proposal. It needs only task labels, the same data the supersample uses, and no repeated training. It may well rank intervention damage as well as leakage memorization does. The first experiment settles this directly by comparing rank correlations with measured intervention damage across the model sweep. If the naive drop matches, we drop the diagnostic claim and keep the decomposition only as an explanation of why leakage hurts. The penalty does not depend on winning this comparison, because the hardening drop is not differentiable training signal that separates memorized from generalizable residual information.
Why not just use hard concepts, or Hard AR, and give up the residual? Hard bottlenecks pay whenever the attribute vocabulary is not sufficient for the label, a limitation both Koh et al. and Margeloiu et al. acknowledge. Margeloiu et al. themselves add a side channel for this reason. Our method competes on exactly that margin. If Hard AR already sits within seed noise of the soft model on CUB and MIMIC-III, the room for improvement is small. The CUB + Places shift setting, where concepts are less complete relative to background cues, is where we expect the gap to be largest.
Why not just regularize the side channel with standard capacity limits like width, dropout, or weight decay? These limit how much information the residual carries, but not which information. They would cut generalizable signal along with memorized signal. The width-limited and shuffled-critic ablations test this. If either matches the residual critic, the train/ghost signal is not what matters and we will say so.
Why not just apply RIB to the whole representation? RIB was evaluated only on uninterpretable latents. Penalizing the supervised concepts would also fight the concept loss. The whole-vector ablation measures this cost in concept accuracy and intervention gain.
Proposed Experiments
The main benchmark is CUB-200-2011 with 312 attributes. CUB + Places from Koh et al. tests a shifted setting where attribute incompleteness matters. MIMIC-III early warning score from Margeloiu et al. provides a non-image check. OAI is included if access allows. No reported result exists on Lacuna for RIB or for any loss-difference CMI estimate on concept models or on these datasets, so all such numbers are new measurements.
The first decisive check is diagnostic. Across roughly twenty configurations spanning $\lambda$, training regime, and side-channel width, we compute the Spearman correlation between leakage memorization and intervention damage. Intervention damage is defined as the accuracy lost, or gain foregone, at a fixed intervention budget relative to the hard model. We compare this correlation against the three naive diagnostics above, with bootstrap intervals over supersample draws. Following Information-theoretic generalization bounds for black-box learning algorithms, which found single-sample supersamples both tightest and cheapest, we use $m=1$. We also report how many draws of $U$ are needed before the estimate stabilizes, since the repeated training is the method's main cost.
The second check tests the bound itself. We verify that the total loss-difference information bounds the measured gap non-vacuously for these architectures, as it did for ResNets in Wang and Mao.
The third check tests the method. We compare task accuracy, concept accuracy, and full intervention curves for the residual-penalized model against Hard AR, joint soft, whole-vector RIB, and the ablations, all under matched data and tuning budgets.
The result that would most change how interpretable models are built is the following. The leakage-memorization term is large, it predicts intervention damage better than the hardening drop, and penalizing it alone closes most of the accuracy gap to the end-to-end model without losing intervention gains. The clearest negative outcome is a small conditional term alongside large damage. That would show leakage is a generalizable shortcut, and the field should look to richer concept vocabularies rather than regularizers.
References
- Addressing Leakage in Concept Bottleneck Models
- Tighter Information-Theoretic Generalization Bounds from Supersamples
- Information-theoretic generalization bounds for black-box learning algorithms
- Concept Bottleneck Models
- Class-wise Generalization Error: an Information-Theoretic analysis
- Recognizable Information Bottleneck