Models that explain their predictions through human-readable concepts are often said to cost little accuracy. A concept bottleneck model is one example: it first predicts named attributes such as "has burner" or "metal surface," then predicts the label from those attributes with a simple linear rule. We think that average-case claim hides a geographic pattern. A model's concept vocabulary is written, and its concept detectors are trained, mostly on the kinds of objects found in wealthy Western homes. The cost of forcing predictions through that vocabulary should therefore be smallest where the vocabulary fits and largest in low-income and non-Western households. There, a "stove" is a charcoal pit or a three-stone hearth that none of the listed concepts describe. If this holds, the accuracy price of interpretability is a fairness quantity, and the standard way to recover accuracy under shift, adding unnamed residual dimensions, buys accuracy back by giving up transparency exactly for the populations the model already serves worst.
This note proposes two things. First, we measure the accuracy price of interpretability per region and per income quartile on geographically diverse benchmarks, and split it into two parts: the concept detectors failed to see what is there, or the vocabulary has no word for what is there. Second, we repair the gap with named, region-specific "realization concepts." These are human-verifiable visual descriptions of how a shared function, such as "holds fire for cooking," is physically built in a given setting. They are added from a handful of labeled target images and replace opaque residual channels. The goal is to show that the geographic accuracy gap of an interpretable model can be closed without giving up interpretability, and to show which part of the gap each intervention closes.
Core Claim
We define the interpretability price for a group $r$ (a region or an income quartile) as
$$\Delta_r = \mathrm{Acc}_r(\text{black-box probe}) - \mathrm{Acc}_r(\text{concept model}),$$
where both models read the same frozen image features. The black-box probe is a linear classifier on raw features; the concept model is a linear classifier on concept scores computed from those features. Because the backbone is shared, $\Delta_r$ isolates the cost of the bottleneck itself.
We claim three things. First, $\Delta_r$ is substantially larger for the lowest-income quartile of Dollar Street and for African and Southeast Asian images in GeoDE than for the highest-income or European groups. Second, most of that excess comes from missing vocabulary rather than miscalibrated detectors. Third, adding a small number of named realization concepts, organized under shared functional concepts, shrinks the excess $\Delta_r$ more than CONDA's unnamed residual dimensions do per added dimension, while every dimension stays human-readable.
Grounding in Prior Work
Two benchmarks motivate the proposal. The Dollar Street Dataset photographs household objects across 63 countries with household income attached. It shows pretrained classifiers reach 18.0% top-5 accuracy on the lowest income quartile versus 57.4% on the highest, and it names the phenomenon we build on: an object's function is universal while its visual form varies with income. GeoDE commissions balanced photos from six regions and finds CLIP and ImageNet models about 7% worse on African than European images, with classes like "dustbin" at 37.3%. Both evaluate only black boxes, so neither can say whether the failure lies in perception or in the decision rule. WILDS formalizes geographic shifts such as PovertyMap and FMoW and shows invariance methods rarely beat plain training, but it also tests only black boxes.
Three papers supply the method. Concept Bottleneck Models introduces the $x \to c \to y$ structure and test-time intervention: correcting two concepts on knee X-rays cut error from above 0.4 to about 0.3. We reuse intervention as a diagnostic instrument rather than as a user feature. Replacing predicted concepts with audited true ones tells us how much of a region's gap is detector failure. Towards Explaining Distribution Shifts uses sparse optimal transport to explain how a distribution moved and reports a "percent explained" score. We borrow that idea in a lighter form, attributing a regional shift in concept-score space to a few named concepts, but we explain the classifier's failure rather than only the data shift. Diverse Concept Proposals for Concept Bottleneck Models shows that greedy selection recovers more distinct valid concept sets than clustering (up to 6 of 7 on a synthetic task). We use greedy selection to choose which candidate realization concepts to add.
One paper constrains the design. CONDA adapts concept models under shift using unlabeled target data. It shows concept models can fall behind black boxes under shift and recovers up to 28% on Waterbirds, partly through a residual bottleneck of unconstrained dimensions. This is the closest existing work, and it frames the tension we address. CONDA assumes a concept means the same visual thing everywhere and patches incompleteness with unnamed dimensions. We assume concepts are visually different across geographies and patch incompleteness with named ones. CONDA was also never evaluated on geographic data, so its behavior there is itself an open measurement. This Looks Like That, whose prototype networks match or beat their black-box backbone on birds (76.1% vs 74.6%), suggests a second interpretable family worth testing. Its prototypes are projected onto training patches, so with Western-skewed training data they can only depict Western forms, which is the same vocabulary gap in visual form.
Research Program
The model uses a frozen CLIP image encoder. Concept scores are cosine similarities between the image embedding and text embeddings of concept phrases, and a sparse linear head maps those scores to labels. This avoids dense concept annotation, which neither benchmark provides.
The vocabulary has two levels. Functional concepts are shared across regions and describe what the object does or contains ("holds fire for cooking," "stores waste"). Realization concepts describe a physical form ("three stones supporting a pot," "woven basket used as bin," "stainless steel pedal bin"). Each realization is attached to one function. The head is constrained so that a class's evidence for a function can come from any of its realizations, which we implement as a max over the realizations' scores. The resulting explanation reads "predicted stove because the function holds fire is present, realized as charcoal brazier."
The base vocabulary is written from source-region images only, to mimic how such vocabularies are normally built. For each functional concept, an annotator lists the forms visible in source-region images.
The diagnostic has three steps. We hold out regions in GeoDE using leave-one-region-out training, and use Dollar Street income quartiles as an external test. For a stratified audit subsample (on the order of 30 images per class per held-out group), annotators mark which base concepts are actually present and, separately, whether the object shows a form the vocabulary lacks. Oracle intervention on the audited concepts gives the detector-failure share of $\Delta_r$. The images flagged as uncovered, together with the accuracy that remains after intervention, give the vocabulary-gap share.
The repair works from $k$ labeled target images per class. We retrieve candidate realization phrases by matching those images against a large pool of descriptive object phrases. We then greedily add the phrases that most improve held-out target accuracy, in the style of the diverse-proposals paper. A human checks each addition before it is accepted. The budget of added concepts is matched exactly to the number of residual dimensions CONDA is allowed.
Baselines
The naive reference is zero-shot CLIP with plain class-name prompts, which is the model GeoDE reports as 7% worse on Africa than Europe. A second reference is our concept model with the base vocabulary and no repair, which gives the uncorrected $\Delta_r$.
The strongest accuracy competitor is a black-box model trained with the same target access: a CLIP linear probe, and also a fully fine-tuned ResNet, trained on source regions plus the same $k$ labeled target images per class. Dollar Street reports that fine-tuning a ResNet on Dollar Street reaches roughly 75% accuracy across all income levels (The Dollar Street Dataset). That setting uses far more target data than ours, so we treat it as a ceiling and do not expect to match it. All models get the same images, the same feature extractor where applicable, and the same hyperparameter search budget, with model selection done on source-region validation only. Beating this baseline does not mean exceeding its accuracy. It means the repaired concept model's worst-group $\Delta_r$ falls to the level of the best-served group's $\Delta_r$, so interpretability no longer costs the poorest households more.
The strongest interpretability-preserving competitor is CONDA, run with all three components on unlabeled target batches. We also run a CONDA variant given the same $k$ labeled images, so that label access is equal. We found no reported result on Lacuna for CONDA on GeoDE or Dollar Street. The comparison is accuracy at matched numbers of added dimensions, together with the fraction of added dimensions a human rater can name. Beating CONDA means equal or better worst-group accuracy with all added dimensions named.
Two ablations isolate the key ingredient. The first adds the same number of concepts chosen at random from the phrase pool, which tests whether gains come from capacity alone. The second adds realization phrases to a flat vocabulary without the function hierarchy, which tests whether the shared-function structure matters.
Why not just…?
Why not just fine-tune a black box on diverse data? On accuracy, this is the best option when target data is plentiful. Dollar Street's roughly 75% across income levels shows it, and GeoDE's augmentation gains of 9% on Dollar Street and 21% on GeoDE confirm it. Our proposal does not dispute that. The question it answers is different: whether an interpretable model can be made to serve every region equally, and what the gap consists of. GeoDE's $0.87-per-image collection cost is also why we work in the few-shot regime. The matched-$k$ black-box baseline settles how much accuracy, if any, the bottleneck costs at equal data.
Why not just prompt CLIP with region-aware class descriptions? This is the real threat. If the retrieved realization phrases are simply appended to class prompts as an ensemble, they may recover the same accuracy with no bottleneck at all. We run exactly this control. If it matches our model, we conclude that the accuracy gain comes from the vocabulary, not the bottleneck. That would still support the diagnostic claim that the gap is a vocabulary gap. The bottleneck's remaining value would then be intervention and per-concept attribution, which we measure separately with oracle intervention on the audited subset.
Why not just use CONDA's residual dimensions? They may recover accuracy as well or better, because they are unconstrained. If so, the comparison at matched dimension counts shows how much accuracy naming costs in the worst-served regions. That quantifies the trade-off directly rather than resolving it, and it is still a useful finding.
Why not just inspect black boxes with saliency maps? Heatmaps show where a model looked, not which decision rule failed. Saliency also shows architecture-driven center bias even in untrained networks (When Random Saliency Looks Trained), so it cannot separate "did not see the burner" from "saw it but has no rule for it."
Proposed Experiments
The first decisive check is whether $\Delta_r$ is geographically non-uniform at all. If the base concept model's price is the same for the lowest and highest Dollar Street quartiles, and for Africa and Europe in GeoDE, the central premise fails and the project reduces to a CONDA evaluation. The second check is the decomposition on audited images. If oracle intervention closes most of the excess gap, the problem is detector calibration, and alignment methods like CONDA's concept-score correction are the right tool rather than new vocabulary. The third check is the repair curve: worst-group accuracy and named-fraction as functions of $k$ (1, 5, 20 images per class) and of the added-concept budget, compared against CONDA, the prompt-ensemble control, and the two ablations.
We report results on GeoDE leave-one-region-out splits and on Dollar Street income quartiles. As a stretch test of transfer to non-household imagery, we also run on FMoW-WILDS, where functional land-use concepts map naturally onto the hierarchy. A secondary analysis applies sparse transport in concept-score space between source and held-out regions. It reports what fraction of the shift a small set of named concepts explains, which checks whether the concepts we add are the ones that actually moved.
References
- The Dollar Street Dataset: Images Representing the Geographic and Socioeconomic Diversity of the World
- GeoDE: a Geographically Diverse Evaluation Dataset for Object Recognition
- WILDS: A Benchmark of in-the-Wild Distribution Shifts
- Concept Bottleneck Models
- Towards Explaining Distribution Shifts
- Diverse Concept Proposals for Concept Bottleneck Models
- CONDA: Adaptive Concept Bottleneck for Foundation Models Under Distribution Shifts
- This Looks Like That: Deep Learning for Interpretable Image Recognition
- When Random Saliency Looks Trained: Architectural Center Bias in CNN Interpretability