TL;DR
CONDA (CONcept-based Dynamic Adaptation) is a framework that makes interpretable Concept Bottleneck Models (CBMs) robust to real-world distribution shifts. By dynamically adapting concept vectors and prediction layers using only unlabeled test data, CONDA boosts post-deployment accuracy by up to 28% while ensuring that model explanations remain relevant to the new environment.
What Problem Does This Paper Address?
Foundation Models (FMs) like CLIP are powerful but operate as "black boxes," making them difficult to trust in safety-critical fields like healthcare or autonomous driving. Concept Bottleneck Models (CBMs) attempt to solve this by forcing the model to first predict human-understandable "concepts" (e.g., "wings," "beak") before making a final classification.
However, CBMs are notoriously brittle. If the test data differs from the training data—due to sensor noise (low-level shift) or changes in background/context (concept-level shift)—the pre-defined concepts often fail to generalize. These shifts cause significant performance drops, sometimes making interpretable models perform worse than their "black-box" counterparts.
Methods
The authors propose CONDA, a three-step test-time adaptation procedure that updates the model "on the fly" using unlabeled target data:
- Concept-Score Alignment (CSA): This step adjusts the concept vectors to ensure that the distribution of concept scores in the new domain matches the expected distribution from the training data. It uses a Mahalanobis distance-based loss to handle low-level input perturbations like noise.
- Linear Probing Adaptation (LPA): Here, the model updates the final "label predictor" layer. This allows the model to re-learn which concepts are actually relevant for classification in the new environment, helping it ignore spurious correlations (like a "water" background always implying a "waterbird").
- Residual Concept Bottleneck (RCB): If the original concept set is incomplete for the new domain, CONDA introduces a small set of "residual" concepts. These are learned jointly with a linear predictor to capture missing information and improve accuracy.
To guide these updates without ground-truth labels, CONDA uses pseudo-labeling, creating an ensemble of zero-shot and linear-probing predictions from the underlying foundation model.
Datasets
The framework was evaluated across diverse real-world scenarios:
- CIFAR10-C & CIFAR100-C: Standard benchmarks for low-level image corruptions (noise, blur, etc.).
- Waterbirds: A classic dataset for concept-level shift where birds are often seen on "spurious" backgrounds (e.g., landbirds on water).
- Metashift: Simulates disparate correlations, such as cats appearing on sofas in the source but on shelves in the target.
- Camelyon17: A medical imaging dataset featuring natural shifts caused by different hospital equipment and lighting.
Key Insights and Ideas
- Interpretability is Dynamic: A model's explanation is only useful if it's accurate for the current data. CONDA shows that "fixing" the interpretation at test-time actually helps the model's performance.
- Different Shifts, Different Fixes: The authors found that CSA is crucial for low-level noise, while LPA and RCB are the heavy lifters for semantic shifts like background changes.
- Residuals Fill the Gaps: Pre-defined concept banks (often generated by LLMs like GPT-3) are rarely perfect. The "Residual" branch allows the model to discover task-relevant features that humans might have missed during initial setup.
Key Results
- Massive Accuracy Gains: On the Waterbirds dataset, CONDA improved the accuracy of a deployed CBM by 28%, bringing it on par with non-interpretable models.
- Better Explanations: Qualitative analysis shows that before adaptation, the model might rely on spurious concepts (like "land" for a landbird). After CONDA, the model correctly shifts its weight to more robust, class-relevant concepts (like "beak" or "feathers").
- Efficiency: Despite performing optimization at test-time, CONDA is fast, taking only a few seconds per batch on standard hardware.
Limitations and Future Work
- Pseudo-label Dependency: CONDA's success relies on the underlying Foundation Model being "mostly right." If the FM is completely baffled by a new domain, the pseudo-labels will be inaccurate, and adaptation will fail.
- Hyperparameter Sensitivity: While the authors provide a heuristic for choosing the number of residual concepts, finding the optimal balance between accuracy and interpretability remains a challenge.
- Future Directions: The authors suggest exploring more advanced pseudo-labeling techniques and developing theoretical bounds to quantify how much a model can realistically adapt to a new domain without source data.
Summary updated