Domain Specific Polysemy in Language Models

38 concepts36 papers4 proposals

Consider the word "spring." To a general-purpose language model trained on millions of web pages, "spring" most likely evokes a season, a coiled piece of metal, or a freshwater source. But drop that same model into a programming forum like StackOverflow, and "spring" suddenly refers to a ubiquitous Java framework. As Jeniya Tabassum demonstrates in Code and Named Entity Recognition in StackOverflow, general-purpose natural language processing models frequently stumble here because common English words double as highly specific technical identifiers.

This phenomenon is known as domain-specific polysemy. It occurs when words possess both a common, everyday meaning and a highly specialized technical meaning within a professional, scientific, or industrial field. It represents a fundamental blind spot for modern language models: high general linguistic intelligence does not automatically translate into domain expertise.

The Mechanism of Semantic Interference

Why do large language models (LLMs) struggle so intensely with these shifts? The answer lies in a cognitive and statistical hurdle called semantic interference.

When a model encounters a word like "charges" in a financial document, its internal representations are tugged in two directions. The vast majority of its pre-training data associates "charges" with legal indictments or physical energy. In a specialized financial context, however, it means expenses or fees. As Mariam Nakhlé et al. show in DOLFIN - Document-Level Financial Test-Set for Machine Translation, even state-of-the-art models frequently default to the common general meaning, erroneously translating financial "charges" as "criminal charges."

This isn't just a machine problem; it mirrors human cognitive biases. In A Laypeople Study on Terminology Identification across Domains and Task Definitions, Anna Hätty and Sabine Schulte im Walde discovered that the familiarity of a word's common usage actively interferes with a person's ability to recognize it as a technical term. For both humans and models, the dominant general-language sense creates a layer of semantic noise that obscures the specialized meaning.

The Three Faces of Domain Ambiguity

Domain-specific polysemy manifests in three distinct ways across technical literature:

  1. Everyday vs. Technical Overlap: Common words take on precise industry roles. For example, Amila Silva et al. note in JPLink: On Linking Jobs to Vocational Interest Types that models routinely misinterpret job requirements because they fail to separate everyday language from professional jargon. Similarly, Nadeesha Pathirana et al. find in Concept Discovery through Information Extraction in Restaurant Domain that models conflate technical industry categories with everyday objects.
  2. Cross-Domain Conflicts: A word's meaning can change violently when moving between two different technical fields. Weimin Fu et al. illustrate this in Hardware Phi-1.5B: A Large Language Model Encodes Hardware Domain Specific Knowledge, where a term like "interface" might mean a software protocol in computer science but a physical hardware pin in semiconductor engineering.
  3. Acronyms and Shorthand Overloading: Technical fields rely heavily on compressed tokens. Hiroya Takamura points out in Metric-Type Identification for Multilevel Header Numerical Tables in Scientific Papers that a single abbreviated label or character can represent entirely different metrics depending on the scientific subfield. Resolving these requires deep contextual awareness, a challenge also explored by Taiqiang Wu et al. in Prompt-based Model for Acronym Disambiguation via Negative Sampling.

Why Architectural Scaling Alone Fails

When moving from general domains (like Wikipedia) to specialized ones (like law or medicine), models suffer severe performance drops. Saptarshi Sengupta et al. demonstrate in Exploring Language Model Generalization in Low-Resource Extractive QA that this gap isn't merely due to missing facts, but rather an architectural inability to resolve domain-specific word senses during zero-shot inference.

Because general-purpose models are optimized to predict the most statistically probable token across a broad web corpus, they are fundamentally biased toward the majority sense. Without explicit intervention, a model reading a telecommunications standard will misinterpret overloaded acronyms (Omar Erak et al., Leveraging Fine-Tuned Retrieval-Augmented Generation with Long-Context Support: For 3GPP Standards), and a clinical model will misclassify homonyms (Reza Averly and Xia Ning, Entity Decomposition with Filtering: A Zero-Shot Clinical Named Entity Recognition Framework).

Strategies for Disambiguation

To overcome semantic interference, researchers have developed several distinct strategies that force models to prioritize local and structural context over global pre-training biases:

By shifting the model's focus from global token frequencies to structured, domain-specific context, these methods prevent general linguistic knowledge from overriding technical precision.

Key Foundational Work

A General Semantic Analyser for Data Base Access (1981) — contributed 1 concept elements to this direction.
Unsupervised Domain Relevance Estimation for Word Sense Disambiguation (2004) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_domain_specific_polysemy_aa35ba57,
  title = {domain-specific polysemy},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/domain-specific-polysemy/rdr_aa35ba57c6dd4eaab04279849d38154e},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_domain_specific_polysemy_aa35ba57_v1,
  title = {domain-specific polysemy},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/domain-specific-polysemy/rdr_aa35ba57c6dd4eaab04279849d38154e/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake