Lexicon Driven Curation for Domain Specialization

This direction is split into other directions: domain-specific stopword removal. Its last version stays readable and citable here.

Version 1 of 1238 papers256 conceptsfrozen 2026-10-04

Modern machine learning faces a "needle in a haystack" problem. While we have access to massive, general-purpose datasets like CommonCrawl, Wikipedia, and Twitter, these sources are often too diluted for specialized tasks. A model trained on the entire internet might know a little about everything, but it often lacks the depth required for niche fields like telecommunications, ophthalmology, or remote sensing.

To bridge this gap, researchers have converged on a strategy of lexicon-driven curation: using targeted keywords, regular expressions, and ontologies to filter massive, unlabeled corpora into high-signal, domain-specific training sets.

The Efficiency of Targeted Filtering

The core intuition behind this approach is that domain-specific knowledge is often signaled by a unique vocabulary. By identifying these "anchor terms," researchers can isolate relevant data from millions of unrelated entries without the prohibitive cost of manual labeling.

In the realm of task-oriented dialog, Anne Lauscher, Goran Glavaš, and their colleagues developed DS-TOD. Their framework specializes models by mining domain-specific data from unlabeled collections using representative keywords. This targeted specialization allows modular "adapters" to outperform general pretraining while using significantly less computational power.

Similarly, for highly technical fields, Hang Zou and the team behind TelecomGPT used a predefined set of telecom-specific terms to extract relevant subsets from general datasets. This allowed an 8-billion parameter model to outperform GPT-4 on specialized tasks like 3GPP document classification.

Why Filtering Works

Filtering does more than just reduce dataset size; it increases the "signal density" of the training material. When a model is exposed to a curated dataset where every sample is relevant, it doesn't have to waste capacity learning to ignore irrelevant noise.

  1. Noise Suppression: In software engineering, Weighted Data Set Reduction techniques strip away non-informative artifacts like console logs and metadata. This ensures the learning algorithm focuses on the core semantic content of bug reports.
  2. Semantic Gating: In clinical settings, Jianbo Jiao and Andrew Zisserman introduced Show from Tell, which uses a domain-specific dictionary to "gate" transcribed speech. By prioritizing anatomically relevant terms, the model avoids learning from the "filler" talk common in medical narrations.
  3. Logical Density: Soumya Sanyal and Yichong Xu demonstrated this with APOLLO, which filters Wikipedia for "logically rich" sentences containing connectives like "therefore" or "however." This simple filtering step significantly boosted the model's reasoning capabilities without needing complex synthetic data.

From Keywords to Ontologies

While simple keyword matching is powerful, the field is moving toward more sophisticated filtering mechanisms that use structured knowledge.

For vertical search engines, researchers have used ontology-driven filtering to extract training examples. By selecting only queries and documents that contain terms found in a specialized medical ontology, they created high-quality training sets for biomedical search without requiring expert human annotations.

In the vision-language domain, Adrian Bulat and Yassine Ouali took this a step further with HC-VL. They used a Large Language Model to generate a hierarchical taxonomy of domain-specific keywords, which then acted as a filter for a massive image-text corpus. This ensured the training data was concentrated on human activities while maintaining the visual diversity of the original source.

Large-Scale Empirical Analysis

Beyond training models, these filtering techniques enable researchers to perform "social listening" at an unprecedented scale. By applying regular expressions and keyword filters to millions of app reviews or social media posts, researchers can pinpoint specific user grievances or public health trends.

By treating the vast expanse of the internet not as a single dataset, but as a collection of overlapping specialized corpora, lexicon-driven curation allows us to build models and conduct studies that are both broad in scope and deep in expertise.

Papers in this version

Showing 60 of 238 papers.

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_lexicon_driven_curation_for_domain_speci_2ea29820,
  title = {Lexicon Driven Curation for Domain Specialization},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/lexicon-driven-curation-for-domain-specialization/rdr_2ea29820155040749a0df159f49db4bf},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_lexicon_driven_curation_for_domain_speci_2ea29820_v1,
  title = {Lexicon Driven Curation for Domain Specialization},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/lexicon-driven-curation-for-domain-specialization/rdr_2ea29820155040749a0df159f49db4bf/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}