Extracting Clusters of Specialist Terms from Unstructured Text

Aaron GerowEMNLP 2014 · DOI

Introduction

In linguistics, the lexicon is often conceptualized as having a "core-periphery" structure. The core consists of high-frequency, ubiquitous words used to define other terms, while the periphery—or "satellites"—contains specialized, domain-specific terminology. Traditional Natural Language Processing (NLP) methods, such as Latent Dirichlet Allocation (LDA), often struggle to isolate these peripheral terms because they prioritize high-frequency words to establish broad topics. Furthermore, many existing terminology extraction methods rely on manually curated dictionaries or extensive linguistic pre-processing.

This paper introduces a graph-based method to automatically extract clusters of these "satellite" terms from raw, unstructured text. By identifying communities in co-occurrence networks and discarding the dominant "core," the authors reveal highly coherent, domain-specific terminology groups without the need for stop-word lists or complex normalization.

Conceptual model of the lexicon's core-periphery structure, illustrating the relationship between the core and satellite terms.

Methodology

The proposed pipeline leverages network theory to identify semantic structures through several key steps:

  1. Graph Construction: A co-occurrence graph is built where nodes represent words and edges denote the frequency of words appearing within a symmetric 7-word window.
  2. Minimal Pre-processing: Unlike standard NLP pipelines, this method does not remove stop-words or perform stemming. The hypothesis is that the network topology itself will segregate functional "noise" from specialized content.
  3. Community Detection: The authors employ the Infomap algorithm, an information-theoretic approach that optimizes a "map" of information flow to find natural clusters.
  4. Core Removal: The algorithm typically identifies one massive, monolithic community representing the core lexicon. By discarding this core (which often comprises 90-97% of the nodes), the researchers focus on the smaller, specialized "satellite" communities.
  5. Intra-community Ranking: To identify representative terms within a cluster, various centrality measures were tested. Hub-score was found to be the most effective for identifying semantically "typical" words.

Evaluation and Results

The method was evaluated across three diverse corpora: TASA (high school level texts), NIPS (scientific proceedings), and enTenTen (a massive web-crawled corpus).

Semantic Coherence

A key finding is that satellite clusters identified via Infomap are consistently more semantically coherent than topics generated by traditional LDA models. Using the UMass Coherence metric, the authors demonstrated that their graph-based approach outperformed a 20-topic LDA model across all three datasets.

Comparison of semantic coherence between the proposed Infomap method and LDA across the enTenTen, NIPS, and TASA datasets.

Centrality and Qualitative Success

Among various centrality measures, Hub-score proved superior to PageRank and simple degree centrality in identifying the most relevant terms within a cluster. Qualitatively, the model successfully isolated distinct thematic groups, such as Middle English vocabulary, Latin American geography, and specialized neuroscientific terms. Statistical validation confirmed that words in satellite clusters have significantly lower frequencies than those in the core, proving the method effectively targets the "lexical fringe."

Discussion and Future Work

The research confirms that the core-periphery structure observed in formal dictionaries is a fundamental property of natural, unstructured discourse. By treating the high-frequency "core" as noise, specialized structures become immediately visible.

Future research directions include exploring how the size of the core varies based on corpus specialization and developing "mixed membership" models where a single word can belong to multiple clusters. Additionally, diachronic analysis could track how terms migrate from the periphery to the core over time as they enter general usage.

Summary updated

BibTeX
@inproceedings{gerow2014extracting,
  author = {Aaron Gerow},
  title = {{Extracting Clusters of Specialist Terms from Unstructured Text}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2014},
  doi = {10.3115/v1/d14-1149},
}
Figure