When humans read a sentence like "The giant swallowed the boulder," we instantly recognize it as a plausible, if fantastical, event. If we read "The boulder swallowed the giant," we know it is semantically bizarre. Syntactically, both sentences are identical: Subject - Verb - Object. Yet our brains enforce a set of semantic constraints on what kinds of entities can perform or receive specific actions.
In computational linguistics, these semantic expectations are known as selectional preferences. Modeling selectional preferences statistically allows natural language processing (NLP) systems to build a form of "common sense." By learning which nouns typically pair with which verbs, models can resolve structural ambiguities, fill in missing information, and correctly map out "who did what to whom."
Overcoming Sparsity: From Words to Semantic Classes
The simplest way to model selectional preference is to count how often specific words appear together in a corpus. However, language is incredibly sparse. If a model has never observed the exact phrase "swallow a rambutan," a pure word-counting approach might assume the phrase is impossible. To build true intuition, a model must generalize.
To solve this lexical sparseness problem, Philip Resnik introduced a class-based approach in A Class-Based Approach to Lexical Discovery. Instead of computing the statistical affinity between a verb and a specific noun, Resnik mapped nouns to broader semantic classes within a taxonomic hierarchy (such as WordNet). By measuring how much a verb changes the distribution of semantic classes in its argument slots compared to the overall distribution of nouns, the model quantifies the verb's "selectional strength."
This abstraction allows a system to recognize that because "rambutan" belongs to the Food class, it fits perfectly as the object of "swallow". This capability is especially powerful in data-scarce environments. For example, Barbara McGillivray demonstrated in Automatic Selectional Preference Acquisition for Latin Verbs that clustering arguments into semantic classes allows meaningful selectional preferences to be extracted even from small, ancient corpora, successfully ranking unseen but valid arguments highly.
[Exact Words] eat -> rambutan, eat -> durian (Highly Sparse)
│ │
▼ ▼
[Semantic Class] eat -> FOOD (Generalizes to all food items)
Rather than relying on rigid, hand-built taxonomies, modern approaches use data-driven probabilistic frameworks to discover these categories automatically. Alan Ritter et al. applied Latent Dirichlet Allocation (LDA) to this problem in A Latent Dirichlet Allocation Method for Selectional Preferences. By treating selectional preferences as a generative process, the authors modeled a full probability distribution over argument-relation pairs, allowing the system to filter logical inferences by checking if they satisfy learned semantic constraints.
Similarly, Diarmuid O Seaghdha et al. utilized directional conditional probabilities in Probabilistic models of similarity in syntactic context to capture how a word's meaning shifts depending on its syntactic environment. Inverting this relationship, Dirk Hovy developed verb-centric graphical models in How Well can We Learn Interpretable Entity Types from Text?, treating the verb as the primary conditioning variable to automatically discover domain-specific, human-interpretable entity types like QUARTERBACK or SEDATIVE based purely on the structural constraints verbs impose on their arguments.
Resolving Ambiguity and Filling the Blanks
Why do we care about a verb's "taste" in nouns? Because natural language is riddled with structural and contextual ambiguities that syntax alone cannot resolve. Selectional preferences act as a tie-breaker.
Syntactic Disambiguation
Consider languages with flexible word orders or ambiguous case markings. In German, a noun phrase can often serve as either the subject or the object. To resolve this, Erika F. de Lima developed a statistical back-off model in Assigning Grammatical Relations with a Back-off Model that calculates the independent probability of noun-verb pairings. If a noun is statistically much more likely to be an object for a given verb, the model leverages this semantic tendency to assign the correct grammatical relation.
A similar problem occurs in English with "reduced passive" phrases (e.g., "the man arrested" could mean the man performed the arrest or was the one arrested). In Learning to Identify Reduced Passive Verb Phrases with a Shallow Parser, Sean Igo et al. used empirical data to estimate whether a noun is statistically more likely to act as an agent or a theme for a specific verb, resolving the active/passive ambiguity through thematic role probabilities.
Recovering Omitted Arguments (Zero-Anaphora)
In languages like Japanese, speakers frequently omit subjects or objects when they are clear from context. To perform zero-anaphora resolution, models must guess the missing word. Kenji Imamura et al. tackled this in Discriminative Approach to Predicate-Argument Structure Analysis with Zero-Anaphora Resolution by integrating generative language model scores that estimate the semantic likelihood of a noun filling a specific case slot.
As deep learning matured, Tomohide Shibata et al. transitioned this to neural architectures in Neural Network-Based Model for Japanese Predicate Argument Structure Analysis, training networks on large unannotated corpora to learn these semantic associations as a form of common sense. To make these models even more resilient, Shuhei Kurita et al. introduced adversarial training in Neural Adversarial Training for Semi-supervised Japanese Predicate-argument Structure Analysis, forcing the model to internalize the linguistic expectations of predicates so it can accurately identify omitted pronouns.
Coreference and Contextual Resolution
Selectional preferences also serve as a vital external knowledge source for resolving pronouns and figurative language:
- Pronoun Resolution: When a sentence reads "The trophy didn't fit in the suitcase because it was too big," syntax cannot tell us if "it" is the trophy or the suitcase. Hongming Zhang et al. demonstrated in Incorporating Context and External Knowledge for Pronoun Coreference Resolution that analyzing large-scale external corpora for the statistical likelihood of noun-verb pairings provides the common-sense signal needed to clear up such ambiguities.
- Bridging Anaphora: In A Rule-Based System for Unrestricted Bridging Resolution: Recognizing Bridging Anaphora and Finding Links to Antecedents, Yufang Hou et al. resolved indirect links (e.g., linking "the astronauts" back to "the space shuttle") by measuring the frequency of specific anaphor-preposition-antecedent triplets in massive text corpora.
- Metonymy Resolution: When a phrase uses a location to represent an entity (e.g., "Washington negotiated a treaty"), Vivi Nastase et al. showed in Combining Collocations, Lexical and Encyclopedic Knowledge for Metonymy Resolution that calculating whether a grammatical context prefers a literal location or a semantic class (like
OrganizationorPerson) reveals whether a word is being used literally or figuratively.
Mapping the Structure of Actions (Semantic Role Labeling)
The ultimate application of selectional preference modeling is Semantic Role Labeling (SRL)—the task of parsing a sentence into a structured format that answers who, what, where, and when.
[Sentence] "The chef cooked a delicious meal in the kitchen."
│ │ │ │
[SRL Mapping] Agent Verb Theme Adjunct (Location)
Before a model can classify semantic roles, it must first identify which parts of a sentence are actually arguments. In Unsupervised Argument Identification for Semantic Role Labeling, Omri Abend et al. proved this can be done without human-annotated data by calculating the Pointwise Mutual Information (PMI) between a verb and a constituent. This relies on the intuition that true semantic arguments appear with their predicates far more frequently than random chance suggests.
The authors extended this logic in Fully Unsupervised Core-Adjunct Argument Classification (Omri Abend et al.), using selectional preferences to separate core arguments (essential participants required by the verb, like the thing being cooked) from adjuncts (optional modifiers, like the location of the cooking) by measuring the strictness of the verb's semantic requirements.
Once arguments are identified, they must be classified. To prevent models from failing when encountering unfamiliar words in new domains, Beñat Zapirain et al. demonstrated in Generalizing over Lexical Features: Selectional Preferences for Semantic Role Classification that computing semantic compatibility via a second-order distributional similarity measure allows systems to generalize role classifications to unseen words.
The authors further refined this in Improving Semantic Role Classification with Selectional Preferences (Beñat Zapirain et al.) by mapping words to broader semantic categories and distributional contexts, evaluating how well a candidate concept type fits within a verb's expected semantic constraints to significantly boost classification accuracy.
Scaling to Higher-Order Eventualities
Traditional selectional preference modeling operates at the word level, evaluating the relationship between a single predicate and a single noun. Recent breakthroughs have scaled this concept up to entire events, activities, and states—collectively termed eventualities.
In ASER: A Large-scale Eventuality Knowledge Graph, Hongming Zhang et al. constructed a massive knowledge graph containing hundreds of millions of eventualities. By quantifying the statistical likelihood of specific entity-action pairings at scale, ASER serves as a backbone for commonsense reasoning, allowing models to evaluate the plausibility of complex real-world scenarios.
[Traditional] Verb [cook] ── prefers ──► Noun [meal]
[Higher-Order] Event [being hungry] ── prefers (Causality) ──► Event [making a call]
Taking this a step further, Xin Liu et al. introduced higher-order selectional preferences over eventualities in ASER: Towards Large-scale Commonsense Knowledge Acquisition via Higher-order Selectional Preference over Eventualities. Instead of just asking what noun a verb prefers, this mechanism asks what event a given event prefers, mapping out the statistical likelihood of discourse relations like causality, condition, or contrast between event pairs.
By treating the statistical distribution of these higher-order linguistic units as a proxy for world knowledge, these systems capture human experiential preferences, transforming selectional preference modeling from a tool for local syntactic parsing into a scalable framework for global commonsense understanding.
Key Foundational Work
A Class-Based Approach to Lexical Discovery (1992) — contributed 1 concept elements to this direction.
Syntax, Preference, and Right Attachment (1985) — contributed 1 concept elements to this direction.