Asymmetric Semantic Associations in Language Models

40 concepts32 papers1 proposal

In standard natural language processing, we often treat the relationship between two words as a "two-way street." If we measure the similarity between doctor and hospital using a metric like cosine similarity, the result is the same regardless of which word we start with. However, human language and cognition are fundamentally directional. While the word stork might immediately evoke the concept of a baby, the word baby does not necessarily trigger stork with the same intensity.

This research direction focuses on asymmetric semantic association: the idea that the strength of a connection from a cue word to a response word is not reciprocal. By moving beyond symmetric geometric models, researchers aim to better reflect the directional structure of human thought and improve how machines understand context, metaphors, and hierarchies.

The Cognitive Mismatch of Symmetry

Traditional models of semantic distance often rely on symmetric metrics like cosine similarity or Euclidean distance. As Washtell and Markert argue, these measures fail to account for the directional nature of human cognitive lexicons, where one concept may strongly trigger another, but the reverse is not true.

This asymmetry is particularly visible in how we categorize information. Schickel and Faltings point out that generalizing from a specific concept (like durian) to a broad category (fruit) is not cognitively equivalent to specializing in the opposite direction. Similarly, West et al. demonstrate through human navigation patterns in Wikipedia that specific instances strongly trigger general categories, but general categories do not necessarily trigger specific instances.

Even our perception of similarity is biased by prototypes. Campbell and Zhang highlight that humans more naturally say "an ellipse is like a circle" than "a circle is like an ellipse," because the circle serves as a geometric prototype. Models that ignore this directionality miss a crucial layer of how humans organize meaning.

Mechanisms for Modeling Directionality

To capture these non-reciprocal relationships, researchers have moved away from simple distance metrics toward mathematical formulations that inherently support asymmetry.

1. Conditional Probability and Influence

The most direct way to model asymmetry is through conditional probability: $P(\text{target}|\text{context}) \neq P(\text{context}|\text{target})$. V. Kripásundar proposed Directed Influence Measures (DIMs) to quantify how one word specifically predicts another, finding them more effective for disambiguation than symmetric mutual information scores. More recently, Yao et al. developed WordTies, which uses constrained sampling in Large Language Models (LLMs) to measure the probability of a response word appearing given a specific cue.

2. Input vs. Output Vector Spaces

A clever structural trick involves the internal architecture of word embeddings like Word2Vec. During training, these models maintain two distinct vectors for every word: one for when the word is the "center" (input) and one for when it is the "context" (output). Torabi Asr et al. and Lin et al. demonstrate that by calculating the similarity between the input vector of a cue and the output vector of a target, models can successfully capture asymmetric associations that standard symmetric comparisons would miss.

3. Information Theory and Fuzzy Sets

Rather than treating words as points in space, some researchers treat them as distributions or sets. Bhat et al. propose treating word embeddings as tuples of feature probabilities. This allows the use of asymmetric measures like Kullback–Leibler (K-L) divergence or cross-entropy, which naturally account for the directional relationship between a specific concept and a broader one.

Applications in Linguistic Structure

Capturing asymmetry isn't just a theoretical exercise; it provides a more nuanced toolset for complex NLP tasks.

  • Metaphor and Sentiment: Lin et al. use the asymmetry between center and context words to identify literal substitutes for metaphors. In sentiment analysis, Zhai et al. use conditional layer normalization to distinguish the directional roles of tokens, such as identifying which word is the source and which is the target of a sentiment link.
  • Parsing and Multiword Expressions: Nguyen et al. reframed constituency parsing as a "pointing" task, where the relationship from a start index to an end index is distinct from the reverse. In the study of multiword expressions (like "loan shark"), Farahmand and Henderson found that the modifier’s ability to predict the headword is a more reliable indicator of a stable expression than the reverse.
  • Cross-Domain Reasoning: When AI explains recommendations by linking different domains (e.g., algebra to visual diagrams), Stockdill et al. show that the strength of an inference from one domain to another is rarely equal to its reverse.

By embracing the "one-way street" of language, these models move closer to the experiential and hierarchical way humans process information, filling the gaps left by traditional, symmetric representations.

Key Foundational Work

Quantifying Lexical Influence: Giving Direction to Context (1995) — contributed 1 concept elements to this direction.
Quantitative Approximations to the Word (1969) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_asymmetric_semantic_association_e8073ff9,
  title = {asymmetric semantic association},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/asymmetric-semantic-association/rdr_e8073ff973274ea4bd12b1f2be76cd77},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_asymmetric_semantic_association_e8073ff9_v1,
  title = {asymmetric semantic association},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/asymmetric-semantic-association/rdr_e8073ff973274ea4bd12b1f2be76cd77/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake