When a computational model processes the word "apple," it leverages a dense web of predictable linguistic contexts, physical attributes, and clear hierarchical relationships. An apple is a fruit, it can be eaten, it is red or green, and it fits neatly into structured taxonomies. However, when the same model encounters the word "justice" or "humility," its performance routinely degrades.
This performance gap is not merely a consequence of data scarcity. Instead, it stems from a fundamental structural asymmetry between concrete and abstract concepts. Across tasks ranging from semantic tagging and relation extraction to sentiment analysis and text classification, natural language processing (NLP) models consistently struggle with abstract language. Understanding why this asymmetry exists requires looking at how abstract concepts break the core assumptions underlying modern computational semantics.
Contextual and Distributional Variance
Most modern semantic models operate on the distributional hypothesis: words that appear in similar contexts share similar meanings. For concrete nouns, this works remarkably well because physical objects are constrained by the real world, leading to highly predictable grammatical and situational contexts.
Abstract nouns, by contrast, are characterized by imperceptibility and nonspecificity. As James R. Curran demonstrated in Supersense Tagging of Unknown Nouns Using Semantic Similarity, abstract categories like relations or cognition lack highly constrained or predictable grammatical patterns. Because an abstract word can appear in a vast, highly varied array of linguistic contexts, its distributional signature becomes diffuse and overlapping.
This contextual overlap creates severe challenges for automated classification systems. As noted by Massimiliano Ciaramita et al. in Supersense Tagging of Unknown Nouns in WordNet, semantic classification systems struggle to distinguish between abstract categories precisely because they lack distinct contextual signaling. When linguistic contexts overlap too heavily, statistical models cannot easily draw clean decision boundaries, leading to lower accuracy in downstream tasks like corporate risk detection (Jiaxin Pei et al. in Modeling and Detecting Company Risks from News).
The Geometry of Semantic Space
Because concrete and abstract words behave differently in text, they organize themselves into entirely different geometric patterns within vector spaces.
In Semantic Similarity Computation for Abstract and Concrete Nouns Using Network-based Distributional Semantic Models, Elias Iosif et al. revealed that concrete words tend to form tight, localized clusters of close synonyms. Abstract words, however, relate to broad, sprawling constellations of attributes. For example, while "car" tightly clusters with "vehicle" and "automobile," "justice" connects diffusely to "law," "fairness," "court," and "morality."
Concrete Space: Abstract Space:
[automobile] [law] ---- [morality]
| | /
[car] -- [vehicle] | [justice]
| \
[court] ---- [fairness]
Standard similarity metrics often fail to account for these structural variations. While a "maximum sense" approach works well for navigating the tight clusters of concrete nouns, capturing the meaning of abstract nouns requires modeling attributional similarity across wider semantic neighborhoods. When models attempt to map abstract qualities or subjective impressions using frameworks designed for concrete objects, the resulting semantic structures are often unstable or overly complex (S. Nishida et al. in Behavioral correlates of cortical semantic representations modeled by word vectors).
The Human Consensus Ceiling
Supervised machine learning models are only as good as the data used to train them. For abstract concepts, this data is inherently noisy because humans themselves struggle to agree on the boundaries and definitions of non-physical entities.
When building lexical resources or annotating corpora, researchers measure inter-annotator agreement to ensure data quality. Across languages and frameworks, abstract categories consistently yield significantly lower agreement scores than concrete ones. In Coarse Lexical Semantic Annotation with Supersenses: An Arabic Case Study, Nathan Schneider et al. observed this drop-off when annotators tried to categorize cognitive processes or general actions versus physical objects.
This lack of human consensus creates a natural performance ceiling for computational models. As Philipp Cimiano et al. argued in Automatic Acquisition of Ranked Qualia Structures from the Web, concrete objects have observable physical properties that humans universally recognize, making automated extraction highly effective. Abstract nouns lack these stable, universally agreed-upon semantic structures, meaning that the "gold standards" used to train and evaluate models are fundamentally more ambiguous.
Missing Hierarchies and Lexical Anchors
Traditional lexical databases like WordNet rely heavily on inheritance hierarchies (e.g., an oak is a tree, which is a plant, which is an organism). This structural design is highly effective for concrete entities but breaks down for abstract concepts.
Abstract concepts often lack clear hierarchical definitions or unique lexical markers. For instance, Davide Buscaldi et al. found in Classification of keyphrases from scientific publications using WordNet and word embeddings that high-level abstract categories, such as research tasks or goals, are significantly harder to classify than materials or processes because they lack formal, structured definitions in traditional databases.
Furthermore, abstract concepts lack the visual features and pragmatic anchors that ground everyday language. As the authors of Refer-to-as Relations as Semantic Knowledge pointed out, while we can easily map formal scientific taxonomies of physical objects to common vernacular (e.g., mapping basidiomycete to mushroom), extending these frameworks to abstract concepts remains an unsolved challenge. Without visual features or rigid associative pathways, abstract concepts are highly resistant to strategic conversational traps or automated alignment (Yuan Yao 0013 et al. in Adversarial Language Games for Advanced Natural Language Intelligence; Adriana Roventini et al. in Mapping Concrete Entities from PAROLE-SIMPLE-CLIPS to ItalWordNet: Methodology and Results).
Bridging the Gap
To resolve abstract meaning, models cannot rely solely on surface-level linguistic features or simple lexical hierarchies. They require deeper pragmatic, situational, or world knowledge (Alla Rozovskaya et al. in UIUC: A Knowledge-rich Approach to Identifying Semantic Relations between Nominals).
Recent strategies focus on forcing models to learn the subtle boundaries of abstract terms through targeted training regimes. For example, in ZJUKLAB at SemEval-2021 Task 4: Negative Augmentation with Language Model for Reading Comprehension of Abstract Meaning, Xin Xie et al. introduced Negative Augmentation with Language Models (NAL). By generating "hard" negative examples—words that are contextually similar but semantically distinct—the authors forced the model to move beyond superficial pattern matching and develop a more nuanced differentiation of abstract meanings.
Ultimately, addressing the concrete-abstract asymmetry requires moving away from the assumption that a single mathematical framework can uniformly represent all types of human thought. By recognizing that abstract concepts are structurally distinct—characterized by diffuse contexts, lower human consensus, and fluid boundaries—researchers can design architectures better suited for the full complexity of human language.
Key Foundational Work
Conceptual Lexicon Using an Object-Oriented Language (1986) — contributed 3 concept elements to this direction.
An Assessment of Semantic Information Automatically Extracted From Machine Readable Dictionaries (1991) — contributed 1 concept elements to this direction.