In natural language processing, not all nouns are created equal. While modern models can identify a "dog" or a "chair" with high precision, they frequently stumble when encountering words like "justice," "theory," or "suitability." This persistent performance disparity is known as the concreteness gap. Across tasks ranging from semantic tagging to relation extraction, computational models consistently find physical, perceptible entities easier to categorize and relate than their abstract counterparts.
The Predictability of the Physical
The primary reason for this gap lies in how words appear in context. Concrete nouns tend to inhabit highly constrained and predictable grammatical environments. As James R. Curran demonstrates in Supersense Tagging of Unknown Nouns Using Semantic Similarity, physical objects like "artifacts" or "locations" possess distinct "semantic signatures." Because these words are used in specific ways—you sit on a chair, you walk into a building—distributional models can easily map their roles.
Abstract nouns, however, lack these rigid patterns. A word like "cognition" or "relation" can appear in a vast variety of contexts, making its distributional profile "noisy" and difficult for a model to pin down. This variability makes it challenging to identify synonyms or categorize unknown abstract terms using simple vector-space similarity.
Relational Stability and Anchors
The difficulty extends beyond identifying single words to understanding how words interact. In UIUC: A Knowledge-rich Approach to Identifying Semantic Relations between Nominals, Alla Rozovskaya and colleagues found that physical relations, such as a container and its contents, are significantly easier to automate than abstract relations like those between a theme and a tool.
This is because concrete objects serve as "conversational anchors." In the "Adversarial Taboo" game developed by Yuan Yao 0013 in Adversarial Language Games for Advanced Natural Language Intelligence, researchers found that physical objects have specific, functional associations that make them easy to induce or describe. Abstract concepts, being "imperceptible" and "nonspecific," lack these rigid associative pathways, making them resistant to both strategic conversational traps and automated classification.
The Human Consensus Ceiling
A fundamental bottleneck for training models on abstract concepts is that even humans struggle to agree on them. Philipp Cimiano and Johanna Wenderoth explored this in Automatic Acquisition of Ranked Qualia Structures from the Web. When asked to define the "qualia" (core attributes) of a noun, human annotators show high agreement for concrete objects (e.g., a "knife" is for "cutting").
For abstract nouns, this consensus evaporates. If humans cannot agree on the defining features of "freedom" or "value," computational models cannot find a stable "gold standard" to learn from. This lack of consensus creates a natural performance ceiling; as Salud M. Jiménez-Zafra notes in A Multi-lingual Annotated Dataset for Aspect-Oriented Opinion Mining, aspects like "location" are trivial for systems to identify, while abstract aspects like "business suitability" rely on nuanced language that varies wildly between speakers.
Strategies for Bridging the Gap
To overcome these challenges, researchers are moving beyond simple lexical features toward more sophisticated internal representations:
- Synthetic Feature Generation: Yiwen Chen 0004 used generative models to create synthetic semantic features in Synthetic Textual Features for the Large-Scale Detection of Basic-level Categories in English and Mandarin, significantly improving the detection of categories that traditional measures like "cue validity" miss.
- Negative Augmentation: In ZJUKLAB at SemEval-2021 Task 4, Ningyu Zhang used "hard" negative examples—predictions the model itself got wrong—to force the system to learn the subtle, non-lexical boundaries between abstract meanings.
- Silver Standard Data: Varada Kolhatkar addressed the lack of labeled data for "shell nouns" (abstract words like fact or issue) by training models on easier "cataphoric" examples—where the meaning follows the noun—to resolve the more difficult "anaphoric" cases where the noun points backward to complex previous text.
By recognizing that abstract language requires deeper world knowledge and pragmatic reasoning, these researchers are beginning to close the gap between how AI perceives the physical world and how it understands the world of ideas.