Concept Understanding in Large Language Models: An Empirical Study

Jiayi Liao, Xu Chen, Lun DuOpenReview 2023 · Submission to Tiny Papers @ ICLR · OpenReview

While Large Language Models (LLMs) have demonstrated superior comprehension and expressiveness across a wide range of tasks, a critical question remains: do they truly "understand" the world, or are they merely mimicking statistical patterns? This empirical study reveals a significant blind spot in current models: they are notably better at understanding concrete objects (like "animals" or "furniture") than abstract ideas (like "fairness" or "logic"). By testing models on a newly constructed dataset, researchers demonstrate that even as models scale up, they consistently struggle to map the hierarchical relationships of abstract concepts.

The Challenge of Concept Understanding

In real-world applications, tasks require varying levels of abstraction. A reliable AI must be able to classify physical entities just as well as it reasons about human emotions or abstract logic. To investigate whether LLMs handle these two types of concepts equally well, this study focuses on hypernym discovery—a classic NLP task that tests the ability to identify if one word represents a broader category of another (e.g., recognizing that "furniture" is a hypernym of "bed").

Methodology and the D-Concept Dataset

To measure this comprehension, the authors introduced a new benchmark called D-Concept, derived from WordNet, a massive lexical database of semantic relations. WordNet's existing hierarchy explicitly divides nouns into an "abstract" branch and a "physical" (concrete) branch. The resulting dataset contains 10,000 samples, split into training, validation, and testing sets. Each sample is a pair of entities, where positive samples are actual hypernym/hyponym pairs at various distances in the WordNet tree, and negative samples are randomly paired.

The researchers evaluated six prominent pre-trained LLMs: BERT, T5, CLIP, OpenClip, GPT Embedding (text-similarity-ada-001), and ChatGPT.

Two different evaluation approaches were utilized:

  1. Linear Probing: For most models, the internal weights were "frozen," and a simple linear layer was added on top to perform binary classification (determining if a pair is a hypernym or not).
  2. Prompting: For ChatGPT, a zero-shot prompt was used ("Is {entity1} the hypernym of {entity2}?"), alongside definitions to help the model disambiguate the words.

Key Insights: The Abstraction Gap

Across every single model tested, performance on abstract concepts was significantly weaker than on concrete ones. LLMs appear to have a much harder time placing abstract ideas within a logical hierarchy.

Interestingly, scaling isn't a silver bullet. While larger models (like the GPT series) perform better overall than smaller ones (like BERT), the performance gap between abstract and concrete concepts persists. Scaling improves general accuracy but does not necessarily resolve the underlying difficulty with abstraction.

Furthermore, T-SNE visualizations of the models' embedding spaces reveal that models naturally cluster abstract and concrete concepts into two distinct groups.

T-SNE visualization of abstract and concrete concepts

This spatial distinction suggests that the models "know" these are fundamentally different types of concepts, yet they still struggle to map the internal logic and hierarchical relationships within the abstract cluster.

Quantitative Results

The quantitative results from the hypernym discovery task clearly highlight this disparity across all tested architectures.

Performance comparison of models on abstract vs. concrete concepts

Key takeaways from the results include:

  • GPT Embedding Model: Achieved a strong 91.98% accuracy on concrete concepts, but this performance dropped to 89.45% on abstract ones.
  • ChatGPT (Zero-shot): Exhibited the most dramatic gap, achieving 78.98% accuracy for concrete concepts compared to only 64.13% for abstract concepts. Its F1 score for abstract concepts was particularly low (44.10), indicating a severe struggle to identify these relationships without specific task training.
  • Consistency Across Architectures: Vision-language models like CLIP and OpenClip followed the exact same trend, consistently performing 3–4 percentage points worse on abstract data.

Limitations and Future Work

While this study provides compelling evidence of an abstraction gap, it focuses solely on hypernym discovery. Though a classic NLP benchmark, it represents only one facet of true semantic "understanding."

Additionally, the evaluation methods varied slightly by necessity—ChatGPT was tested via zero-shot prompting, while other models were fine-tuned with a linear probe. While this makes a direct "apples-to-apples" comparison between ChatGPT and BERT difficult, the internal gap between abstract and concrete performance remains undeniably clear within each model's individual results.

Future research must investigate why this semantic gap exists in the embedding spaces of large language models and explore new training techniques designed to help LLMs better grasp high-level, non-physical concepts.

Summary updated

Abstract

Large Language Models (LLMs) have demonstrated their superior comprehension and expressiveness across a wide range of tasks, and exhibited remarkable capabilities in real-world applications. Hence, it is crucial to investigate their potential and limitations for trustworthy performance in both academia and industry. In this paper, we focus on exploring LLMs' ability to understand concepts, especially abstract and concrete ones. To this end, we construct a WordNet-based dataset containing a subset for abstract concepts and a subset for concrete concepts. We select six pre-trained LLMs and conduct a classic NLP task, hypernym discovery, as evidence of LLMs' comprehension ability in understanding concepts. The experimental results suggest that the LLM's understanding of abstract concepts is significantly weaker than that of concrete concepts.
BibTeX
@misc{liao2023concept,
  author = {Jiayi Liao and Xu Chen and Lun Du},
  title = {{Concept Understanding in Large Language Models: An Empirical Study}},
  year = {2023},
  url = {https://openreview.net/forum?id=losgEaOWIL7},
  howpublished = {OpenReview},
  note = {Submitted to Tiny Papers @ ICLR},
}
Figure