When computational linguists build dictionaries, train word embeddings, or design algorithms to induce word senses, they face a silent but pervasive bottleneck: the part-of-speech scope limitation. Historically, natural language processing (NLP) models and lexical resources have disproportionately favored nouns. Nouns are often treated as the default entry point for semantic modeling because they map cleanly to concrete entities, stable concepts, and straightforward taxonomic hierarchies.
However, language is not merely a collection of static objects. Verbs, adjectives, and prepositions carry the relational, structural, and aspectual weight of communication. When a system optimized exclusively for nominal data encounters these dynamic word classes, its underlying assumptions often break down. Understanding why this part-of-speech bias occurs—and how it constrains generalizability—is essential for building truly robust language models.
The Noun Bottleneck in Lexical Resources
Building a comprehensive lexical database requires significant manual or semi-automated effort. To make the task manageable, researchers frequently restrict their initial scope to nouns, intending to expand to other word classes later.
For instance, Elena V. Paducheva, Ekaterina V. Rakhilina, and Marina V. Filipenko developed LEXICOGRAPHER, an expert system designed to bridge computational linguistics and traditional lexicography by treating a lexicon as a relational database in Semantic dictionary viewed as a lexical database (coling 1992). While the system successfully predicted syntactic regularities from word meanings, its initial implementation focused strictly on concrete nouns, leaving its ability to capture broader linguistic phenomena across verbs and pronouns unfulfilled.
This nominal constraint persists across modern resource generation. In Goidelex: A Lexical Resource for Old Irish, Cormac Anderson et al. introduced an interoperable lexical database to unify fragmented historical data, yet the initial release was restricted entirely to nominal lexemes. Similarly, when Yang Liu et al. designed algorithms to map semantic knowledge cross-lingually in Building a Bilingual WordNet-Like Lexicon: The New Approach and Algorithms, they focused primarily on nouns, leaving the mapping of verbs and adjectives as an open challenge.
The reason for this structural preference lies in the nature of semantic relations. Nouns organize neatly into inheritance trees via is-a (hypernymy) relationships (e.g., a robin is a bird, which is an animal). Verbs and adjectives do not fit into these clean hierarchies; they are defined by entailment, aspect, troponymy, and scalar variations, making them significantly harder to formalize.
Why Distributional Models Fail to Generalize
The bias is not confined to manual databases; it is deeply embedded in how we evaluate unsupervised semantic models. Distributional similarity models operate on the principle that words occurring in similar contexts share similar meanings. However, the metrics used to calculate these similarities are often tuned and validated almost exclusively on nouns.
[Noun-Centric Evaluation] ──> High Performance on Concrete Objects (e.g., "apple" ≈ "pear")
│
▼
[Dynamic Word Classes] ──> Fails on Verbs/Prepositions due to Argument Structure & Polysemy
In Characterising Measures of Lexical Distributional Similarity, Julie Weeds et al. investigated how different computational methods handle word frequency and neighbor selection. Crucially, their analysis was limited to nouns and direct-object relationships. When evaluation frameworks are restricted this way, we cannot guarantee that the similarity measures will hold for verbs, which depend heavily on complex subject-object argument structures. This evaluation gap is mirrored in work by Martin Riedl et al. in Combining Supervised and Unsupervised Parsing for Distributional Similarity and by Saif M. Mohammad and Graeme Hirst in Distributional measures of concept-distance: A task-oriented evaluation, where the core testing frameworks relied primarily on nominal data.
When researchers attempt to inject human-interpretable semantics into machine learning models, the part-of-speech barrier reappears. For example, Cindy Aloui et al. introduced SLICE in SLICE: Supersense-based Lightweight Interpretable Contextual Embeddings to map embedding dimensions directly to broad semantic categories ("supersenses"). While highly effective, the approach was restricted to nouns because acquiring adequate seed lists and annotated data for verbs and adjectives proved significantly more challenging.
The Downstream Effects on NLP Tasks
When core algorithms are built on a noun-heavy foundation, downstream applications inherit these limitations. This pattern is evident across several distinct NLP tasks:
Word Sense Induction (WSI)
WSI algorithms automatically discover the distinct meanings of a word from text without relying on a dictionary. In Nonparametric Bayesian Word Sense Induction, Xuchen Yao and Benjamin Van Durme used a Hierarchical Dirichlet Process to automatically determine the number of senses for a given word. However, their evaluation focused entirely on nouns. When Anna Aksenova et al. built a data-driven benchmark for Russian WSI in RuDSI: Graph-based Word Sense Induction Dataset for Russian, they similarly restricted their manual annotations to nouns. Because verbs are highly polysemous and their meanings shift based on syntactic context (e.g., "fire a gun" vs. "fire an employee"), WSI models evaluated only on nouns often fail when deployed in the wild.
Dataset Generation
To train large-scale semantic models, researchers often generate artificial data. Mohammad Taher Pilehvar and Roberto Navigli addressed data sparsity by creating artificial ambiguous words in Paving the Way to a Large-scale Pseudosense-annotated Dataset. Yet, their pseudoword generation technique was limited to nominal words, leaving the modeling of ambiguity in verbs, adjectives, and adverbs unaddressed.
Morphological Processing and Translation
Morphological tools like stemmers are frequently designed with specific word classes in mind. In Extracting Loanwords from Mongolian Corpora and Producing a Japanese-Mongolian Bilingual Dictionary, Badam-Osor Khaltar et al. built a pipeline to identify foreign terms using a dictionary-free stemming algorithm. Because the stemmer was optimized for nouns, it completely excluded loanword verbs, which undergo entirely different morphological transformations.
Even when rules are expanded, they often remain narrow: Hussein Soori et al. developed an Arabic stemming approach for data compression in A Linguistic Method into Stemming of Arabic for Data Compression that targeted only three main parts of speech (nouns, verbs, adjectives), despite their parser identifying 27 distinct grammatical categories.
Breaking Out of the Noun Bias
To overcome these limitations, some researchers have deliberately isolated other parts of speech to build specialized frameworks. While these projects successfully move beyond nouns, they often remain siloed within their own target word class:
- Verbs: Hideki Tanaka et al. tackled "delexical structures" (where verbs like take or give empty themselves of meaning, as in "take an approach") in A Method of Translating English Delexical Structures Into Japanese, but limited their scope to a tiny subset of deverbal nouns and delexical verbs. Similarly, Zdeňka Urešová et al. built the CzEngClass lexicon in Synonymy in Bilingual Context: The CzEngClass Lexicon to capture cross-lingual verb synonymy, and Sreekavitha Parupalli et al. automated verb sense identification for Telugu in Towards Automation of Sense-type Identification of Verbs in OntoSenseNet.
- Prepositions: Closed-class words present their own unique challenges. Huichao Xue and Rebecca Hwa modeled word choice errors for ESL learners in Modeling ESL Word Choice Similarities By Representing Word Intensions and Extensions, but their methodology was strictly tailored to prepositions, making its generalizability to open-class words uncertain.
- Adjectives: In Predicting Cognitively Salient Modifiers of the Constitutive Parts of Concepts, Gerhard Kremer and Marco Baroni developed corpus-based techniques to extract composite properties (like "long ears"), but explicitly restricted their extraction pipeline to adjective modifiers.
The ultimate goal of this research direction is to move away from single-category silos and build unified, cross-categorical models. Language models cannot fully comprehend context if their understanding of verbs is decoupled from their understanding of nouns.
Recent work has begun pushing contrastive learning and deep learning architectures to ingest broader lexical knowledge. For instance, Anna Mosolova et al. demonstrated in Injecting Wiktionary to improve token-level contextual representations using contrastive learning that injecting curated lexical knowledge can significantly improve large language model representations. However, because their training data primarily focused on verb lemmas, the authors noted that it remains an open question whether incorporating all parts of speech simultaneously will yield uniform performance gains.
True generalizability in lexical semantics requires moving past the convenience of noun-only modeling. By designing evaluation metrics, datasets, and architectures that treat verbs, adjectives, and closed-class words as first-class citizens, the field moves closer to capturing the full, interconnected complexity of human language.
Key Foundational Work
Semantic dictionary viewed as a lexical database (1992) — contributed 1 concept elements to this direction.
Working on the Italian Machine Dictionary: A Semantic Approach (1973) — contributed 1 concept elements to this direction.
A Method of Translating English Delexical Structures Into Japanese (1992) — contributed 1 concept elements to this direction.