In text analysis, natural language processing, and computational social science, researchers frequently rely on lexicons—curated lists of keywords or phrases that represent specific concepts, sentiments, or topics. However, static, hand-crafted dictionaries suffer from a fundamental limitation: language is dynamic, highly contextual, and prone to vocabulary drift. A static list of keywords fails when confronted with evolving internet slang, platform-specific jargon, or historical shifts in terminology.
Manually updating these dictionaries is slow, expensive, and scales poorly. Embedding-based lexicon expansion addresses this challenge by leveraging the geometric properties of continuous word vector spaces to automatically discover contextually relevant synonyms, slang, and related concepts from a small set of high-quality "seed" words.
The Core Mechanism: Geometric Proximity
The intuition behind embedding-based expansion relies on the distributional hypothesis: words that appear in similar contexts share similar meanings. When neural network models (such as Word2Vec, GloVe, or skip-gram architectures) are trained on a large corpus, they map words into a high-dimensional vector space where contextual alignment translates directly into spatial proximity.
[Seed Words] ---> Map to Vector Space ---> Compute Cosine Similarity ---> [Expanded Lexicon]
("unsafe") (Dense Embeddings) (Find Nearest Neighbors) ("risky", "hazardous")
To expand a lexicon, a researcher starts with a small set of expert-curated seed terms that define the target construct. The system computes the geometric center (or individual vectors) of these seeds within the embedding space. It then calculates a distance metric—most commonly cosine similarity—between the seed vectors and all other candidate words in the corpus vocabulary. The words with the highest similarity scores are extracted as semantic neighbors and added to the expanded lexicon.
For example, Fast et al. (2016) developed Empath, demonstrating that a small set of user-defined seeds can generate highly accurate lexical categories on demand by identifying nearest neighbors in a high-dimensional space. Similarly, Tsai et al. (2014) showed that financial risk prediction can be enhanced by expanding small sentiment lexicons using continuous word vector representations to find contextually relevant keywords in SEC filings that standard dictionaries omit.
Why Domain Adaptation Matters
A common pitfall in natural language processing is relying on general-purpose embeddings (e.g., trained on Wikipedia or Google News) to expand a specialized lexicon. If a researcher uses general embeddings to expand a lexicon for analyzing clinical notes or online forums, the model will miss the local dialect, abbreviations, and specialized jargon.
To capture true semantic alignment, the embedding space must reflect the specific corpus being analyzed. Mitra et al. (2016) demonstrated that training embeddings on a localized, query-specific corpus significantly outperforms global embeddings for query expansion because it captures the precise nuances of topical language.
In specialized fields, this local training is critical:
- Medicine: Antoine Simoulin et al. trained embedding models on local clinical corpora to discover domain-specific synonyms and abbreviations without relying on pre-existing medical ontologies, making text mining adaptable to local medical dialects.
- Finance: Theil et al. (2018) trained models on billions of words from SEC disclosures to expand uncertainty lexicons, doubling the ability of these tools to explain future stock market volatility compared to static dictionaries.
- Software Engineering: Ruiyin Li et al. started with coarse-grained keywords and expanded them via embeddings to capture how developers discuss architecture erosion in code reviews, uncovering natural language patterns missed by string matching.
Capturing Slang, Stance, and Temporal Drift
Because language shifts rapidly across different platforms and eras, embedding-based expansion serves as a linguistic bridge, mapping formal concepts to informal or historical variants.
Social Media Vernacular
On platforms like Reddit or Weibo, users frequently invent new slang or use platform-specific variants to express ideas. Pavalanathan et al. (2017) expanded seed markers using embeddings trained on massive Reddit datasets to capture platform-specific equivalents of formal linguistic constructs. For tracking adversarial behavior, Srikanth et al. (2019) used embedding models to dynamically discover rapidly evolving keywords used by online trolls, ensuring high recall in fast-moving debates where static lists fail.
Historical Evolution
Over long periods, the core vocabulary used to discuss social issues changes entirely. Kostikova et al. (2024) utilized semantic expansion of seed terms to track the concept of "solidarity" across 155 years of German parliamentary debates. By calculating vector similarities across different historical slices, the authors captured evolving terminology and synonyms that traditional keyword searches would completely miss in archival data.
Robustness Against Missing and Out-of-Vocabulary Terms
Beyond expanding dictionaries for content analysis, this geometric mapping provides a crucial fallback mechanism for downstream machine learning tasks when encountering out-of-vocabulary (OOV) or rare terms.
When an NLP system encounters an unknown word, it can calculate the semantic similarity between that unknown word and known trigger words or seed terms. Jung‐Jae Kim et al. demonstrated this in active learning systems, using embeddings to map unknown words to known semantic clusters, allowing the system to accurately estimate document importance even when processing text with unfamiliar terminology.
Similarly, Abhishek Kumar et al. utilized a distributional thesaurus to identify the most semantically similar neighbors to act as proxies for missing tokens in financial text, allowing models to handle noisy microblog data and rare words seamlessly.