Interpreting Pretrained Language Models via Concept Bottlenecks (Extended Abstract)

Zhen Tan, Lu Cheng, Song Wang, Bo Yuan, Jundong Li, Huan Liu

Introduction

Pretrained Language Models (PLMs) such as BERT and GPT have revolutionized natural language processing, yet they remain largely "black boxes." While standard interpretability methods like attention weights can highlight specific words, they often fail to provide high-level semantic reasoning. For instance, knowing a model focused on the word "lobster" in a restaurant review doesn't clarify whether the sentiment was driven by food quality, price, or service. Concept Bottleneck Models (CBMs) address this by predicting intermediate human-understandable concepts before the final output. However, CBMs have traditionally faced two major hurdles: a high "annotation tax" (requiring manual labeling) and a "performance penalty" (reduced accuracy).

The $C^3M$ Framework

To overcome these barriers, researchers introduced $C^3M$ (ChatGPT-guided Concept augmentation with Concept-level Mixup). This framework automates the creation of interpretable models without sacrificing performance.

The Concept Bottleneck architecture maps input text to high-level concepts (e.g., Food, Ambiance, Service) before predicting the final target label.

The $C^3M$ approach relies on three pillars:

  1. Concept Augmentation: Utilizing LLMs like ChatGPT to brainstorm a comprehensive set of relevant concepts from a few seed examples.
  2. Noisy Label Generation: Using LLMs as automated annotators to generate pseudo-labels for large datasets, bypassing the need for expensive human labeling.
  3. Concept-level Mixup (CM): A novel training strategy that interpolates between data points to mitigate the impact of noisy LLM labels, ensuring the model remains robust and accurate.

The resulting CBE-PLM architecture forces the model's internal representations through a "bottleneck" where each neuron corresponds to a specific human concept.

Interpretable Predictions and Human Intervention

A primary advantage of this framework is the transparency it provides. Instead of a single sentiment score, the model offers a breakdown of how different concepts contributed to the final decision.

An example of a concept-based explanation where the model attributes a high sentiment score to positive food and ambiance, despite negative service.

As shown in the figure above, for the input "Excellent lobster and decor, but rude waiter," the model identifies positive contributions from "Food" and "Ambiance" and a negative contribution from "Service." This transparency enables "test-time intervention": if a model misidentifies a concept, a human can manually correct it, and the model will immediately update its final prediction based on the corrected logic.

Key Results and Performance

Evaluations on the CEBAB (restaurant reviews) and IMDB-C (movie reviews) datasets demonstrate that interpretability does not have to come at a cost. On the CEBAB dataset, the CBE-PLM-CM model achieved a task accuracy of 94.4%, a significant improvement over the standard BERT baseline of 80.5%. Furthermore, the "Mixup" strategy proved vital; it allowed the model to learn effectively from noisy LLM-generated labels, often resulting in higher robustness than models trained on smaller, clean datasets.

Conclusion

The $C^3M$ framework represents a significant step toward making powerful language models more transparent and accountable. By leveraging the general knowledge of LLMs to automate concept discovery and labeling, researchers have shown that we can build models that are both highly accurate and easily interpretable by humans.

Summary updated

Abstract

Pretrained language models (PLMs) achieve state-of-the-art results but often function as ``black boxes'', hindering interpretability and responsible deployment. While methods like attention analysis exist, they often lack clarity and intuitiveness. We propose interpreting PLMs through high-level, human-understandable concepts using Concept Bottleneck Models (CBMs). This extended abstract introduces C3M (ChatGPT-guided Concept augmentation with Concept-level Mixup), a novel framework for training Concept-Bottleneck-Enabled PLMs (CBE-PLMs). C3M leverages Large Language Models (LLMs) like ChatGPT to augment concept sets and generate noisy concept labels, combined with a concept-level MixUp mechanism to enhance robustness and effectively learn from both human-annotated and machine-generated concepts. Empirical results show our approach provides intuitive explanations, aids model diagnosis via test-time intervention, and improves the interpretability-utility trade-off, even with limited or noisy concept annotations. This is an concise version of [Tan et al., 2024b], recipient of the Best Paper Award at PAKDD 2024. Code and data are released at https://github.com/Zhen-Tan-dmml/CBM_NLP.git.
BibTeX
@inproceedings{tan2025interpreting,
  author = {Zhen Tan and Lu Cheng and Song Wang and Bo Yuan and Jundong Li and Huan Liu},
  title = {{Interpreting Pretrained Language Models via Concept Bottlenecks (Extended Abstract)}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.24963/ijcai.2025/1221},
}
Figure